Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
node_modules/
dist/
.turbo/
.DS_Store
*.log
*.tsbuildinfo
89 changes: 89 additions & 0 deletions README.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,3 +145,92 @@ data-recipe/
## License

TBD

## Run the MVP locally

### Install dependencies

```bash
pnpm install
```

### Build the browser extension

```bash
pnpm build:extension
```

The extension output is:

```text
apps/extension/dist
```

### Load it in Chrome

1. Open `chrome://extensions`;
2. Enable Developer mode;
3. Click "Load unpacked";
4. Select `apps/extension/dist`;
5. Click the "AI 有数" extension icon to open the side panel.

### Manual verification

1. Open `docs/test-page.html`;
2. Click "开始发现" in the side panel;
3. Click the fetch or XHR test button on the test page;
4. The side panel should show the discovered data source count, URL, method, status, query data, and response preview;
5. Expand "高级信息" to inspect the minimal Data Recipe draft JSON.

You can also start discovery on any page you are authorized to access and then trigger a page query. This MVP only performs low-frequency local detection. It does not bypass login, captchas, risk controls, or dynamic signatures.

### Development mode

```bash
pnpm dev:extension
```

This watches and rebuilds `apps/extension/dist`. Refresh the unpacked extension in Chrome before testing updated code.

### Export a Data Skill Package

After one discovery flow, use the side panel to:

1. Fill in the data skill name and purpose;
2. Confirm or rename returned fields;
3. Check "试运行结果" to confirm data can be read;
4. Check "技能包预览" to confirm required files are present;
5. Click "导出技能包".

The current MVP downloads a `.data-skill.json` file. This is a temporary text package format containing:

```text
packageName
files[]
SKILL.md
recipe.json
examples.md
README.md
```

A later version can export a real folder or zip. For now, the JSON text package validates the generate, test, and export loop.

### Data Skill Package acceptance checks

An exported package should at least:

* Include `SKILL.md`, `recipe.json`, `examples.md`, and `README.md`;
* Use `SKILL.md` to explain what the data skill is useful for;
* Use `recipe.json` to hold the data source, query inputs, returned fields, and test-run information;
* Show a data preview in the side panel test-run result;
* Show no missing required files before export.

### Validate an exported skill package

After exporting a `.data-skill.json` file, run:

```bash
pnpm validate:skill-package path/to/your.data-skill.json
```

The command checks that all four required files are present and non-empty. It exits with a non-zero status when the package is invalid, so it can be used in later automated checks.
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,19 @@ Turn web page queries and API responses into reusable data recipes for AI agents

* [中文说明](./README.zh-CN.md)
* [English README](./README.en.md)

## MVP quick start

```bash
pnpm install
pnpm build:extension
```

Load `apps/extension/dist` as an unpacked extension in Chrome, open `docs/test-page.html`, open the DataRecipe side panel, click "开始发现", trigger the fetch or XHR test request, then confirm the generated Data Skill Package and export the `.data-skill.json` file.


## Validate an exported skill package

```bash
pnpm validate:skill-package path/to/your.data-skill.json
```
89 changes: 89 additions & 0 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,3 +138,92 @@ data-recipe/
## License

TBD

## 本地运行 MVP

### 安装依赖

```bash
pnpm install
```

### 构建浏览器插件

```bash
pnpm build:extension
```

构建产物在:

```text
apps/extension/dist
```

### 在 Chrome 中加载插件

1. 打开 `chrome://extensions`;
2. 开启「开发者模式」;
3. 点击「加载已解压的扩展程序」;
4. 选择 `apps/extension/dist`;
5. 点击浏览器工具栏里的「AI 有数」图标打开侧边栏。

### 手动验证

1. 打开 `docs/test-page.html`;
2. 在 AI 有数侧边栏点击「开始发现」;
3. 在测试页点击「触发 fetch 查询」或「触发 XHR 查询」;
4. 侧边栏应显示发现的数据来源数量、URL、请求方式、状态、查询条件和返回预览;
5. 展开「高级信息」可以查看最小数据配方草稿 JSON。

也可以在任意你有权访问的网页上点击「开始发现」,然后触发页面查询。当前 MVP 只做低频本地探测,不会绕过登录、验证码、风控或动态签名。

### 开发模式

```bash
pnpm dev:extension
```

该命令会监听并重新构建 `apps/extension/dist`。Chrome 扩展页面中需要手动点击刷新插件后再验证最新代码。

### 导出 Data Skill Package

完成一次发现后,可以在侧边栏中:

1. 为数据技能填写名称和用途说明;
2. 确认或修改返回字段名称;
3. 查看「试运行结果」确认能读取数据;
4. 在「技能包预览」中确认已包含必需文件;
5. 点击「导出技能包」。

当前 MVP 会下载一个 `.data-skill.json` 文件。它是临时的文本包格式,里面包含:

```text
packageName
files[]
SKILL.md
recipe.json
examples.md
README.md
```

后续会把这个导出格式升级为真实文件夹或 zip。当前阶段先用 JSON 文本包验证「可生成、可测试、可导出」的闭环。

### Data Skill Package 验收点

导出的技能包至少应满足:

* 包含 `SKILL.md`、`recipe.json`、`examples.md`、`README.md`;
* `SKILL.md` 能说明这个数据技能适合完成什么任务;
* `recipe.json` 包含数据来源、查询条件、返回字段和测试运行所需信息;
* 侧边栏「试运行结果」可以展示数据预览;
* 导出前没有提示缺少必需文件。

### 校验导出的技能包

导出 `.data-skill.json` 后,可以用下面的命令检查它是否包含四个必需文件,并确认文件内容不为空:

```bash
pnpm validate:skill-package path/to/your.data-skill.json
```

校验通过时会输出技能包名称和文件列表;缺少文件或文件为空时会返回非零退出码,便于后续接入自动化验收。
22 changes: 22 additions & 0 deletions apps/extension/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "@data-recipe/extension",
"version": "0.1.0",
"private": true,
"type": "module",
"scripts": {
"clean": "rimraf dist",
"build": "tsup src/background.ts src/content.ts src/page-hook.ts src/sidepanel.ts --format iife --target es2020 --splitting false --clean --out-dir dist && node scripts/copy-assets.mjs",
"dev": "tsup src/background.ts src/content.ts src/page-hook.ts src/sidepanel.ts --format iife --target es2020 --splitting false --watch --out-dir dist --onSuccess \"node scripts/copy-assets.mjs\"",
"typecheck": "tsc -p tsconfig.json --noEmit"
},
"dependencies": {
"@data-recipe/detector": "workspace:*",
"@data-recipe/recipe-core": "workspace:*",
"@data-recipe/recipe-runner": "workspace:*",
"@data-recipe/skill-builder": "workspace:*",
"@data-recipe/skill-exporter": "workspace:*"
},
"devDependencies": {
"rimraf": "^6.0.1"
}
}
31 changes: 31 additions & 0 deletions apps/extension/public/manifest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{
"manifest_version": 3,
"name": "DataRecipe / AI 有数",
"description": "发现网页背后的数据来源,并生成可复用的数据配方草稿。",
"version": "0.1.0",
"action": {
"default_title": "AI 有数"
},
"background": {
"service_worker": "background.global.js"
},
"content_scripts": [
{
"matches": ["<all_urls>"],
"js": ["content.global.js"],
"run_at": "document_start"
}
],
"side_panel": {
"default_path": "sidepanel.html"
},
"permissions": ["activeTab", "sidePanel", "tabs", "scripting", "storage"],
"host_permissions": ["<all_urls>"],
"web_accessible_resources": [
{
"resources": ["page-hook.global.js"],
"matches": ["<all_urls>"]
}
]
}

Loading