Skip to content

Commit 9655e8f

Browse files
authored
Merge pull request #65 from 1041206149/0401
chore:Code comments & LLM api config
2 parents 79e8191 + 2ef14f7 commit 9655e8f

60 files changed

Lines changed: 2825 additions & 2819 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.env.example

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# LLM Configuration
2+
LLM_BASE_URL=https://api.openai.com/v1
3+
LLM_API_KEY=your-api-key-here
4+
LLM_MODEL=gpt-4o
5+
USE_LLM=True
6+
7+
# Debug
8+
METRICS_DEBUG=False

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,9 @@ coverage.xml
4747

4848
webmainbench.egg-info/*
4949

50+
# cache files
51+
webmainbench/.cache/
52+
5053
# PyPI packaging
5154
build/
5255
dist/

README.md

Lines changed: 11 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -148,31 +148,28 @@ hf_hub_download(
148148
)
149149
```
150150

151+
### Configure LLM (Optional)
152+
153+
LLM-enhanced content splitting improves formula/table/code extraction accuracy. To enable it, copy `.env.example` to `.env` and fill in your API credentials:
154+
155+
```bash
156+
cp .env.example .env
157+
# Edit .env and set LLM_BASE_URL, LLM_API_KEY, LLM_MODEL
158+
```
159+
151160
### Run an Evaluation
152161

153162
```python
154163
from webmainbench import DataLoader, Evaluator, ExtractorFactory
155164

156165
dataset = DataLoader.load_jsonl("data/WebMainBench_545.jsonl")
157-
extractor = ExtractorFactory.create("trafilatura")
166+
result = Evaluator().evaluate(dataset, ExtractorFactory.create("trafilatura"))
158167

159-
evaluator = Evaluator(llm_config={
160-
"use_llm": True,
161-
"llm_base_url": "https://api.openai.com/v1",
162-
"llm_api_key": "sk-xxxxxxxxxxxx",
163-
"llm_model": "gpt-4o",
164-
})
165-
result = evaluator.evaluate(dataset, extractor)
168+
m = result.overall_metrics
166169

167170
print(f"Overall Score: {result.overall_metrics['overall']:.4f}")
168171
```
169172

170-
If you don't need LLM-enhanced content splitting (for formula/table/code extraction), disable it explicitly:
171-
172-
```python
173-
evaluator = Evaluator(llm_config={"use_llm": False})
174-
```
175-
176173
### Compare Multiple Extractors
177174

178175
```python

README_zh.md

Lines changed: 11 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -148,31 +148,28 @@ hf_hub_download(
148148
)
149149
```
150150

151+
### 配置 LLM(可选)
152+
153+
LLM 增强内容拆分可提升公式/表格/代码的抽取精度。如需启用,将 `.env.example` 复制为 `.env` 并填写 API 信息:
154+
155+
```bash
156+
cp .env.example .env
157+
# 编辑 .env,设置 LLM_BASE_URL、LLM_API_KEY、LLM_MODEL
158+
```
159+
151160
### 运行评测
152161

153162
```python
154163
from webmainbench import DataLoader, Evaluator, ExtractorFactory
155164

156165
dataset = DataLoader.load_jsonl("data/WebMainBench_545.jsonl")
157-
extractor = ExtractorFactory.create("trafilatura")
166+
result = Evaluator().evaluate(dataset, ExtractorFactory.create("trafilatura"))
158167

159-
evaluator = Evaluator(llm_config={
160-
"use_llm": True,
161-
"llm_base_url": "https://api.openai.com/v1",
162-
"llm_api_key": "sk-xxxxxxxxxxxx",
163-
"llm_model": "gpt-4o",
164-
})
165-
result = evaluator.evaluate(dataset, extractor)
168+
m = result.overall_metrics
166169

167170
print(f"Overall Score: {result.overall_metrics['overall']:.4f}")
168171
```
169172

170-
如不需要 LLM 增强内容拆分(用于公式/表格/代码抽取),可显式关闭:
171-
172-
```python
173-
evaluator = Evaluator(llm_config={"use_llm": False})
174-
```
175-
176173
### 多抽取器对比
177174

178175
```python

0 commit comments

Comments
 (0)