You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
These are exact copies of test_run/test_output/<case>/eqdata* produced by the standalone eq.x / tot/tot Fortran binaries at the time of commit. They act as the silent-SKIP safety net for Layer 1 (1e-10) equivalence tests in the three Phase L Python wrappers.
Risk
If the upstream eq.x / tot/tot binary's EQSAVE format or numerics drift (e.g. compiler upgrade, eq subroutine refactor), the committed fixtures could go stale silently: the test would PASS against the OLD baseline + OLD fixture, but the staging-path eqdata (which the test prefers when present) would differ. CI staging currently catches "fixture missing" via test -s, but not "fixture stale".
Surfaced as low-probability LOW by in-house code reviewer on PR #199; same risk applies symmetrically to all 3 libs.
Possible remediations (not yet decided)
A. CI checksum gate: at the end of the CI staging step, cmp (or md5) the CI-generated eqdata against the committed fixture; fail loudly on mismatch.
Context
PR #195 (#192) and PR #199 introduced committed
eqdatabinary fixtures under:python/trlib/tests/fixtures/eqdata.{ITER01,TST-2}python/eqlib/tests/fixtures/eqdata.{ITER01,TST-2}python/totlib/tests/fixtures/{eqdata-HT6M,eqdata.demo2014}(PR test(totlib): mirror trlib/eqlib eqdata fallback to fix silent SKIP #199)These are exact copies of
test_run/test_output/<case>/eqdata*produced by the standaloneeq.x/tot/totFortran binaries at the time of commit. They act as the silent-SKIP safety net for Layer 1 (1e-10) equivalence tests in the three Phase L Python wrappers.Risk
If the upstream
eq.x/tot/totbinary'sEQSAVEformat or numerics drift (e.g. compiler upgrade, eq subroutine refactor), the committed fixtures could go stale silently: the test would PASS against the OLD baseline + OLD fixture, but the staging-path eqdata (which the test prefers when present) would differ. CI staging currently catches "fixture missing" viatest -s, but not "fixture stale".Surfaced as low-probability LOW by in-house code reviewer on PR #199; same risk applies symmetrically to all 3 libs.
Possible remediations (not yet decided)
cmp(ormd5) the CI-generated eqdata against the committed fixture; fail loudly on mismatch.workflow_dispatchregen path (see.github/workflows/regen-baselines.yml) so fixtures can be refreshed on a cadence or on demand.Out of scope
metrics.jsonbaselines undertest_run/baselines/are NOT covered by this issue — they're regenerated more rigorously via PR ci+eq+tr: workflow_dispatch baseline regen + graphics-stubs (#197) #198's CI workflow.Source
In-house code reviewer's LOW L2 finding on PR #199 (totlib eqdata fallback polish).