Skip to content

Commit 287b665

Browse files
stack72claude
andauthored
fix: lock promptfoo transitive dependencies to prevent CI breakage (#1115)
## Summary - Add `package.json` + `package-lock.json` in `evals/promptfoo/` to lock the full promptfoo dependency tree, preventing transitive dependency breakage like the `@asamuzakjp/css-color@5.1.5` incident ([#1110 comment](#1110 (comment))) - The eval script now runs `npm ci` before invoking promptfoo instead of `npx -y promptfoo@0.121.3` - Extended `scripts/audit_deps.ts` to scan the npm lockfile for vulnerabilities alongside `deno.lock` - Updated the CI outdated-deps check to read the version from `package.json` via `jq` ## Test Plan - All existing tests pass (4174 passed) - `deno check`, `deno lint`, `deno fmt --check` all pass - Verified `npm ci && npx promptfoo --version` works correctly in `evals/promptfoo/` - Verified audit script picks up packages from both `deno.lock` (99) and `package-lock.json` (984) - Confirmed `css-color@5.1.5` reproduces the `ERR_REQUIRE_ASYNC_MODULE` error and `5.1.6` (locked in our lockfile) does not Closes #1110 --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 6d9bac1 commit 287b665

7 files changed

Lines changed: 13673 additions & 8 deletions

File tree

.github/workflows/ci.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -107,10 +107,10 @@ jobs:
107107
108108
- name: Check for outdated npm dependencies
109109
run: |
110-
pinned=$(grep -o 'promptfoo@[0-9.]*' scripts/eval_skill_triggers_promptfoo.ts | head -1 | cut -d@ -f2)
110+
pinned=$(jq -r '.dependencies.promptfoo' evals/promptfoo/package.json)
111111
latest=$(npm view promptfoo version 2>/dev/null)
112112
if [ "$pinned" != "$latest" ]; then
113-
echo "::warning::promptfoo is outdated: pinned $pinned, latest $latest — update scripts/eval_skill_triggers_promptfoo.ts"
113+
echo "::warning::promptfoo is outdated: pinned $pinned, latest $latest — update evals/promptfoo/package.json and run npm install to regenerate the lockfile"
114114
else
115115
echo "promptfoo $pinned is up to date"
116116
fi

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,3 +32,4 @@ CLAUDE.local.md
3232
resources/deno/
3333
/scripts/dist
3434
evals/promptfoo/results.json
35+
evals/promptfoo/node_modules/

deno.json

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
"version": "0.1.0",
44
"license": "AGPL-3.0-only",
55
"exports": "./main.ts",
6-
"workspace": ["packages/client", "packages/testing"],
6+
"workspace": ["packages/client", "packages/testing", "evals/promptfoo"],
77
"tasks": {
88
"dev": "deno run --unstable-bundle --allow-read --allow-write --allow-env --allow-run --allow-sys main.ts",
99
"test": "deno test --unstable-bundle --allow-read --allow-write --allow-env --allow-run --allow-net --allow-sys",
@@ -60,6 +60,7 @@
6060
".agents/",
6161
"workflows/",
6262
".vault-test-vault/",
63-
"resources/"
63+
"resources/",
64+
"evals/"
6465
]
6566
}

0 commit comments

Comments
 (0)