Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions .github/workflows/npm-package-sweep.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
name: NPM Package Sweep

on:
schedule:
# Nightly advisory signal for package drift. It does not run on PRs.
- cron: '25 5 * * *'
workflow_dispatch:
inputs:
packages:
description: 'Optional comma-separated package specs. Empty uses packages.json.'
type: string
default: ''
limit:
description: 'Maximum packages to sweep. Use 0 for all selected packages.'
type: number
default: 6
strict:
description: 'Fail the workflow if any package fails.'
type: boolean
default: false

permissions:
contents: read

concurrency:
group: npm-package-sweep-${{ github.ref }}
cancel-in-progress: false

env:
CARGO_TERM_COLOR: always
MACOSX_DEPLOYMENT_TARGET: "13.0"

jobs:
compile-packages-sweep:
name: compilePackages sweep
runs-on: ubuntu-latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v6

- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable

- uses: Swatinem/rust-cache@v2
with:
shared-key: "${{ runner.os }}-perry-npm-sweep"
save-if: ${{ github.ref == 'refs/heads/main' }}

- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: '22'

- name: Build Perry release binary
run: cargo build --release -p perry-runtime -p perry-stdlib -p perry

- name: Run npm package sweep
env:
SWEEP_PACKAGES: ${{ github.event.inputs.packages }}
SWEEP_LIMIT: ${{ github.event.inputs.limit }}
SWEEP_STRICT: ${{ github.event.inputs.strict }}
run: |
set -euo pipefail
mkdir -p .npm-sweep-results
flags=(
--perry-bin "$GITHUB_WORKSPACE/target/release/perry"
--out-dir .npm-sweep-results
--history test-compat/npm-sweep-history.csv
--limit "${SWEEP_LIMIT:-6}"
)
if [[ -n "${SWEEP_PACKAGES:-}" ]]; then
flags+=(--packages "$SWEEP_PACKAGES")
fi
if [[ "${SWEEP_STRICT:-false}" == "true" ]]; then
flags+=(--strict)
fi

set +e
python3 test-compat/npm-sweep/run.py "${flags[@]}"
sweep_rc=$?
set -e

cp test-compat/npm-sweep-history.csv .npm-sweep-results/npm-sweep-history.csv
cat .npm-sweep-results/summary.md >> "$GITHUB_STEP_SUMMARY"
exit "$sweep_rc"

- name: Upload sweep artifacts
if: always()
uses: actions/upload-artifact@v7
with:
name: npm-package-sweep-${{ github.sha }}
path: .npm-sweep-results/
if-no-files-found: ignore
retention-days: 90
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ tests/test-*
!tests/test_typed_feedback_runtime_evidence.py
!tests/test_perf_frontier_report.py
!tests/test_test262_focused_report.py
!tests/test_npm_sweep.py
!tests/test_perf_frontier_gate_smoke.sh
!tests/*/
bench_*
Expand Down Expand Up @@ -193,6 +194,12 @@ test-files/.perry-cache/
!/test-compat/test262/
!/test-compat/test262/**
/test-compat/test262/report.json
!/test-compat/npm-sweep/
!/test-compat/npm-sweep/**
!/test-compat/npm-sweep-history.csv
/test-compat/npm-sweep/__pycache__/
/test-compat/npm-sweep/out/
/test-compat/npm-sweep/.npm-sweep-results/
/test-compat/test262/reports/

# Vendored upstream corpora for the compat radars (large; never committed).
Expand Down
1 change: 1 addition & 0 deletions test-compat/npm-sweep-history.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
timestamp_utc,git_sha,package,requested,resolved_version,status,first_failure_line,install_ms,compile_ms,run_ms,total_ms,perry_version,node_version,npm_version
51 changes: 51 additions & 0 deletions test-compat/npm-sweep/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# npm compilePackages sweep

This directory owns the advisory sweep for #805. It is intentionally separate
from `tests/release/packages/`: release fixtures are pinned, deterministic
gates, while this sweep follows current npm package drift and records a trend
signal.

The runner creates a temporary fixture per package, installs the package,
compiles a tiny namespace import with `perry.compilePackages`, optionally runs
the produced binary, and writes:

- `results.json` with one structured row per package
- `results.csv` with the same rows in trend-friendly form
- `summary.md` for GitHub Actions step summaries
- per-package logs under `logs/<package>/`

The scheduled workflow uploads those files as artifacts. Package failures are
expected while compatibility is incomplete, so the workflow is advisory by
default and does not run on pull requests.

## Local usage

Build Perry first:

```sh
cargo build --release -p perry-runtime -p perry-stdlib -p perry
```

Run the default tier:

```sh
python3 test-compat/npm-sweep/run.py \
--perry-bin target/release/perry \
--out-dir .npm-sweep-results \
--history test-compat/npm-sweep-history.csv
```

Run a dry plan without npm or Perry:

```sh
python3 test-compat/npm-sweep/run.py --dry-run --packages nanoid,ms,zod
```

Manual package selection accepts comma-separated specs:

```sh
python3 test-compat/npm-sweep/run.py --packages express@latest,@types/node@latest
```

Use `--strict` only when intentionally turning the sweep into a gate; without
it, package failures are recorded and the runner exits zero.
73 changes: 73 additions & 0 deletions test-compat/npm-sweep/packages.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
{
"schema_version": 1,
"default_limit": 6,
"description": "Prioritized npm packages for the compilePackages compatibility sweep. The first tier mirrors the initial ad-hoc #805 baseline; later tiers keep the issue-body package set ready for wider manual/nightly runs.",
"packages": [
{
"name": "nanoid",
"version": "latest",
"tier": "tier1",
"reason": "small ESM utility baseline"
},
{
"name": "ms",
"version": "latest",
"tier": "tier1",
"reason": "small CommonJS utility baseline"
},
{
"name": "zod",
"version": "latest",
"tier": "tier1",
"reason": "popular TypeScript-first library with deep re-export surface"
},
{
"name": "uuid",
"version": "latest",
"tier": "tier1",
"reason": "hybrid export/default-wrapper coverage"
},
{
"name": "dayjs",
"version": "latest",
"tier": "tier1",
"reason": "prototype-heavy date utility coverage"
},
{
"name": "chalk",
"version": "latest",
"tier": "tier1",
"reason": "callable object plus property surface coverage"
},
{
"name": "express",
"version": "latest",
"tier": "tier2",
"reason": "issue-body server framework target"
},
{
"name": "hono",
"version": "latest",
"tier": "tier2",
"reason": "issue-body ESM server framework target"
},
{
"name": "pino",
"version": "latest",
"tier": "tier2",
"reason": "issue-body logger target"
},
{
"name": "vitest",
"version": "latest",
"tier": "tier3",
"reason": "issue-body toolchain target with larger dependency graph"
},
{
"name": "prisma",
"version": "latest",
"tier": "tier3",
"reason": "issue-body generated-client/tooling target"
}
]
}
Loading