You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Part of #793 (axis 3 — Node.js platform built-in modules with real semantics).
What already exists (committed, not yet CI-wired)
A behavioral parity suite was built and committed over the last week:
test-files/test_parity_<module>.ts — 42 files, one per supported node:* module. Every API in the parity reference becomes a console.log("label:", expr) line under per-section try/catch. The byte-for-byte diff against node --experimental-strip-types IS the per-module gap report. Commits: d0e84724, 08c9004a, 1f2f44e5.
docs/runtime-parity.md — leaf-level inventory of the entire Node + Bun stdlib API surface (~50 modules, ~4,200 rows: | API | Node.js | Bun | Notes |), sourced from the authoritative Node docs + Bun compat table.
docs/runtime-parity-gaps.md — that inventory diffed against Perry's compile-time coverage (manifest + Expr::* HIR variants + js_* FFI exports across perry-runtime / perry-stdlib / 35 perry-ext-* crates). 2,417 true gaps.
scripts/gen_parity_tests.py + scripts/parity-skiplist.toml — regenerate skeletons from the inventory; skiplist documents architecturally-out-of-scope modules (vm/inspector/v8/etc.) with rationale.
run_parity_tests.sh --filter parity_ — runs just the parity matrix; auto-PERRY_ALLOW_UNIMPLEMENTED=1 so unimplemented APIs surface as runtime divergence instead of hard compile errors; strips DEP/Experimental Node warnings from the diff.
This is ~70% of the axis-3 instrumentation #793 is asking for. It is currently run ad-hoc locally and has repeatedly been cleaned out of the working tree between sessions (now committed, so durable).
What this sub-issue adds
Wire it into CI as a tracked, per-module signal — distinct from #794 (per-category thresholds) and #800 (Node's own test corpus):
CI job runs ./run_parity_tests.sh --filter parity_ on every push to main.
Per-module trend artifact — emit a table of module | node_lines | perry_lines | diff_lines | status so a regression in any single module is visible (not buried in a global %). The diff-line count per module is already computed locally; just persist it as a CI artifact + job annotation.
Baseline lock — store the current per-module diff counts as test-parity/parity_matrix_baseline.json; CI fails if any module's diff grows (regression) and prints a celebratory note when one reaches 0 (full byte-for-byte PASS).
Part of #793 (axis 3 — Node.js platform built-in modules with real semantics).
What already exists (committed, not yet CI-wired)
A behavioral parity suite was built and committed over the last week:
test-files/test_parity_<module>.ts— 42 files, one per supportednode:*module. Every API in the parity reference becomes aconsole.log("label:", expr)line under per-section try/catch. The byte-for-byte diff againstnode --experimental-strip-typesIS the per-module gap report. Commits:d0e84724,08c9004a,1f2f44e5.docs/runtime-parity.md— leaf-level inventory of the entire Node + Bun stdlib API surface (~50 modules, ~4,200 rows:| API | Node.js | Bun | Notes |), sourced from the authoritative Node docs + Bun compat table.docs/runtime-parity-gaps.md— that inventory diffed against Perry's compile-time coverage (manifest +Expr::*HIR variants +js_*FFI exports acrossperry-runtime/perry-stdlib/ 35perry-ext-*crates). 2,417 true gaps.scripts/gen_parity_tests.py+scripts/parity-skiplist.toml— regenerate skeletons from the inventory; skiplist documents architecturally-out-of-scope modules (vm/inspector/v8/etc.) with rationale.run_parity_tests.sh --filter parity_— runs just the parity matrix; auto-PERRY_ALLOW_UNIMPLEMENTED=1so unimplemented APIs surface as runtime divergence instead of hard compile errors; strips DEP/Experimental Node warnings from the diff.This is ~70% of the axis-3 instrumentation #793 is asking for. It is currently run ad-hoc locally and has repeatedly been cleaned out of the working tree between sessions (now committed, so durable).
What this sub-issue adds
Wire it into CI as a tracked, per-module signal — distinct from #794 (per-category thresholds) and #800 (Node's own test corpus):
./run_parity_tests.sh --filter parity_on every push to main.module | node_lines | perry_lines | diff_lines | statusso a regression in any single module is visible (not buried in a global %). The diff-line count per module is already computed locally; just persist it as a CI artifact + job annotation.test-parity/parity_matrix_baseline.json; CI fails if any module's diff grows (regression) and prints a celebratory note when one reaches 0 (full byte-for-byte PASS).test_parity_timers_promises, root causeawaiton namespace-member that returns undefined re-enters catch arm infinitely (5.7M lines observed) #712) emitted 5.7M identical lines and the runner's O(n²) bashnormalize_outputburned ~3 hours. The CI job MUST cap per-test stdout (e.g. 50k lines) and the normalizer must be linear, or pathological output DOSes the run.Current baseline (v0.5.910 sweep, for reference)
os,tty,async_local_storage(+argon2,ethersas expected-output).path(diff = 4, blocked only onpath.posix/path.win32sub-namespace methods return undefined (last gap before path PASS) #810 —path.posix/win32sub-namespace).string_decoder,async_hooks,tls,readline.import * as X from "node:X"where Perry has no stdlib backing — strict-import policy (Namespace import for unrecognized node:* sub-paths resolves totrue(TAG_TRUE) #629 / v0.5.835) correctly rejects them; these need real native bindings, not test changes.http(307),fs(207),stream(141),zlib(130),net(115),buffer(93).Trend across sweeps v0.5.713 → v0.5.910: 0 → 5 PASS as #623/#629/#630/#631/#648/#649/#650/#681/#682/#712/#741/#742 landed.
Done when
parity_matrix_baseline.jsonis committed and regressions fail CI.normalize_outputis linear-time.