C-LEARN: residual Ref.vdf numeric divergence after element-cycle-resolution (SAMPLE UNTIL / INIT-:NA: / numeric tail / NaN-vs-NA)

[bpowers]

Summary

Umbrella tracking item for the smaller residual clusters of C-LEARN's numeric divergence from the genuine-Vensim reference output (test/xmutil_test_models/Ref.vdf) that remain after the element-cycle-resolution work. C-LEARN now compiles, runs to FINAL TIME, and matches Ref.vdf within 1% on 96.3% of matched cells (after the per-element-GF mapping fix #589 / commit 61573545 and a :NA:-aware + near-zero-robust comparator in ensure_vdf_results). The decision was made to bank the 96.3% and track the residual: every variable below is excluded from the simulates_clearn gate via the documented EXPECTED_VDF_RESIDUAL exclusion constant, so the test passes on the reconciled set.

The dominant residual cluster (whole variables importing as a literal 0+0 stub) is tracked in a separate issue. This umbrella collects everything else.

Discovered on branch clearn-hero-model, HEAD 8775ae97; element-cycle-resolution AC8.1.

Residual clusters

1. SAMPLE UNTIL / INIT-of-:NA: / target-policy (COP dimension, often cop_developing_b)

A :NA:-arithmetic edge and/or SAMPLE UNTIL semantics. Vensim performs NA-arithmetic yielding e.g. -2*NA where Simlin yields a bare :NA: sentinel or 0:

• last_set_target_year -- Vensim = -1.298e33 (i.e. -2*NA) vs Simlin = 0; equation is INIT(VECTOR SELECT(...))
• last_active_target_year -- INIT with a -6.49e32 (:NA:) guard
• time_from_target_to_ultimate_target -- rel ~ 0.96
• target_emissions_for_rate / ultimate_target_value_from_rate -- SAMPLE UNTIL
• depth_at_bottom -- INIT recurrence
• emissions_with_stopped_growth

This subsumes the 2-variable "NA-mismatch" bucket (spurious :NA:, e.g. last_set_target_year).

2. Genuine ~25% numeric tail (meaningful-value divergence, not near-zero)

A connected chain that diverges by rel ~ 0.247 on substantial values -- needs a numeric root-cause, not comparator tuning:

• im_3_emissions / im_3_ff_co2 / im_3_emissions_vs_rs / relative_emissions_to_equity / relative_emissions_to_equity_target -- e.g. oecd_us @ step 187
• co2eq_gap_closing_percentage -- large rel but tiny absolute (~1.4e-3); a near-zero-ratio percentage, mostly comparator-ish rather than a true divergence.

3. other-NaN (NaN-vs-:NA: representation mismatch)

~9 variables, led by slr_inches_from_2000[deterministic] (sea-level-rise), where Vensim writes literal IEEE NaN for all 251 steps while Simlin writes the :NA: sentinel (-6.49e32). A NaN-vs-:NA: representation mismatch on a deterministic-scenario series.

Why it matters

• Correctness / Vensim fidelity: :NA:-arithmetic (e.g. -2*NA), SAMPLE UNTIL semantics, and NaN-vs-:NA: representation are each places where Simlin's runtime diverges from genuine Vensim on real model logic.
• The ~25% im_3_emissions chain is a meaningful-magnitude divergence (not near-zero noise) and is the most likely to indicate a substantive engine bug rather than a representation/comparator nuance.

Components affected

• src/simlin-engine/src/ -- builtins / SAMPLE UNTIL, :NA:-arithmetic, VDF NaN handling
• Test fixtures: test/xmutil_test_models/C-LEARN v77 for Vensim.mdl, test/xmutil_test_models/Ref.vdf
• src/simlin-engine/tests/simulate.rs -- simulates_clearn; the exact exhaustive list of excluded vars lives in its EXPECTED_VDF_RESIDUAL exclusion constant.

Discovery context

Identified during the C-LEARN hero-model element-cycle-resolution work (branch clearn-hero-model, HEAD 8775ae97), element-cycle-resolution AC8.1. All variables above are excluded from the simulates_clearn gate and tracked here.

Relationship to existing issues (all DISTINCT)

• engine: Vensim :NA: was mishandled as IEEE NaN (should be the finite sentinel -2^109) #586 (:NA: mishandled as IEEE NaN -> finite sentinel -2^109) -- FIXED; established the runtime :NA: sentinel. The clusters here are the remaining numeric residual after that fix: :NA:-arithmetic results (-2*NA), SAMPLE UNTIL, and the VDF NaN-vs-:NA: representation gap. engine: Vensim :NA: was mishandled as IEEE NaN (should be the finite sentinel -2^109) #586's own "Remaining follow-on" (test comparator reconciling sentinel-vs-VDF-0.0) is the comparator now in ensure_vdf_results; this issue is the genuine divergence that comparator does not paper over.
• engine: C-LEARN full simulation blocked by non-macro issues (circular dep, dimension mismatches, unknown dep) -- the macro epic's deferred 'tracked separately' follow-up #559 (C-LEARN compile-blockers) -- now resolved (C-LEARN runs); this is a post-run numeric residual.
• engine: arrayed VECTOR SORT ORDER returns flat global indices instead of per-iterated-slice 0-based ranks #585 (arrayed VECTOR SORT ORDER flat indices) / engine: VECTOR ELM MAP full_source_len has no genuine-Vensim numeric end-to-end test gate (defense-in-depth coverage gap) #579 (VECTOR ELM MAP coverage gap) -- distinct builtins; not the SAMPLE UNTIL / NA-arithmetic / NaN-vs-NA residual.
• Companion: the dominant 0+0-stub lookup/data import cluster is tracked in a separate issue.

[bpowers]

Full residual investigation — measurement methodology + histogram progression

Recording the complete quantitative investigation behind this residual (so it is fully tracked), spanning the diagnostic, the two high-leverage fixes, and the post-comparator re-measure.

Methodology

A diagnostic harness runs C-LEARN through the incremental path (open_vensim → compile_vm → Vm::new → run_to_end → into_results), parses test/xmutil_test_models/Ref.vdf (VdfFile::parse(...).to_results_via_records()), and categorizes every matched (ident, step) cell — matched set = Ref.vdf.offsets ∩ results.offsets — into buckets. Scale: 873,982 matched cells = 3,482 matched series × 251 steps.

Comparator (post-comparator measurement; ensure_vdf_results, commit 8775ae97)

A cell matches if |e − a| ≤ atol + rtol·max(|e|,|a|) with rtol = 0.01 (the cross-simulator 1%) and a per-series absolute floor atol = K_ATOL · max_step|e[ident]|, K_ATOL = 1e-4. Plus: a matched-variable floor max(10, 10%·|vdf vars|); a NaN-skip guard (fail on an entirely-NaN core series, or a global NaN-skipped fraction > 10%); and the :NA:-sentinel reconciliation (Simlin crate::float::NA = -2^109 ≡ Vensim 0; a Simlin :NA: vs a genuinely-nonzero Vensim value is a real mismatch, never silently passed). The engine keeps the :NA: sentinel — only the comparator interprets it.

Histogram progression (three measurements)

Bucket	Baseline (pre-GF-fix)	Post-GF-fix (#589 / 61573545)	Post-comparator (current, 8775ae97)
match	58.8%	59.7%	83.7% (731,777)
:NA:-sentinel / reconciled	12.5%	12.5%	12.5% (108,934)
near-zero (reconciled)	0.1%	24.0%	0.1% (1,129)
genuine divergence	28.0% (1,314 idents)	3.27% (39 vars)	3.12% (27,251 cells / 31 vars)
NA-mismatch (spurious :NA:)	0.08% (14)	0.56%	0.08% (665 / 2 vars)
other-NaN	0.48% (25)	0%	0.48% (4,226 / 9 vars)

The near-zero bucket jumped post-GF-fix because correcting the per-element-GF mapping moved many wrong-value cells to ≈0-where-Vensim-is-≈0; the near-zero-robust comparator then folds those into match (hence near-zero drops back to 0.1% while match rises to 83.7%).

The two high-leverage fixes (28% → 3.12% genuine divergence)

1. Per-element-GF element→dimension-index mapping (engine: per-element graphical functions are mapped by Arrayed-Vec (sorted) position instead of element-name -> dimension index #589, commit 61573545): per-element graphical-function tables were laid out by Equation::Arrayed Vec position (alphabetically sorted by the MDL importer) instead of by element-name → declared dimension index, so for C-LEARN's non-sorted COP dimension every arrayed GF fed the wrong element's table. Fixing it collapsed the population → gdp → per-capita/emissions/HFC/CO2eq cascade (1,314 → 39 diverging vars).
2. :NA:-aware + near-zero-robust comparator (8775ae97): mechanically reconciled the 12.5% :NA:-sentinel bucket (Simlin -2^109 ≡ Vensim 0) and the near-zero bucket, with no numeric change.

Current genuine residual (~3.7%), banked + tracked

• Dominant 0+0 data/graph-lookup-import cluster → engine: C-LEARN data/graph-lookup variables import as 0+0 (zeroed) instead of their lookup/data values #590 (the bulk of the 31 genuine-divergence vars / 27,251 cells; rel = 1.0, whole vars at 0 vs real Vensim values).
• SAMPLE UNTIL / INIT-:NA: / target-policy cluster, the ~25% im_3_emissions numeric tail, and the other-NaN (NaN-vs-:NA:) cluster → this issue (breakdown in the issue body above).

The exact exhaustive list of excluded variables will live in simulates_clearn's EXPECTED_VDF_RESIDUAL constant (the Phase 7 un-stub commit). Lineage: element-cycle-resolution AC8.1; depends on the :NA: sentinel fix #586 and the GF-mapping fix #589.

[bpowers]

Exact EXPECTED_VDF_RESIDUAL membership (measured on clearn-hero-model, un-stubbed simulates_clearn)

The exact exhaustive exclusion set is now baked into EXPECTED_VDF_RESIDUAL in src/simlin-engine/tests/simulate.rs, enumerated by running C-LEARN through the exact ensure_vdf_results per-cell comparator logic (classify_vdf_ident) and collecting every base with >=1 failing cell. Through the hardened (:NA:-aware + near-zero-robust) comparator the residual is 32 failing bases over 154 failing idents (3298 of 3482 matched idents reconcile, ~95.6%).

This issue's clusters cover 15 of those 32 bases. Two were discovered by the exhaustive measurement beyond the original representative lists; recording them here so every excluded base is attributable:

• Cluster 1 (NA-arithmetic): historical_gdp (the bare twin of the engine: C-LEARN data/graph-lookup variables import as 0+0 (zeroed) instead of their lookup/data values #590 historical_gdp_lookup) belongs HERE, not engine: C-LEARN data/graph-lookup variables import as 0+0 (zeroed) instead of their lookup/data values #590. Its non-:NA: cells match Vensim exactly; only its -2*NA cells diverge -- VDF -1.298e33 (-2*NA) vs Simlin's bare :NA: sentinel -6.49e32 (-2^109). Same NA-arithmetic pattern as last_set_target_year et al.
• Cluster 2 (numeric tail): diffusion_flux -- the ocean-diffusion physical variable ([sensitivity, layer]) feeding the SLR sub-model. It diverges on small magnitudes (VDF 0 vs sim ~1e-8 where the per-series abs floor doesn't cover, and ~0.5 rel on ~1e-5 cells). Same meaningful-but-small numeric-tail character as the im_3_* chain.

Cluster 3 (other-NaN, e.g. slr_inches_from_2000) needs NO exclusion entry: the hardened comparator NaN-skips cells where either side is IEEE NaN (Vensim's early-step NaN vs Simlin's :NA:), and the late finite steps match within 1%, so no cluster-3 base reaches the failure set. The representation gap is still tracked here; it is just not gate-excluded.

Cluster-1+2 bases excluded (this issue): last_set_target_year, last_active_target_year, time_from_target_to_ultimate_target, target_emissions_for_rate, ultimate_target_value_from_rate, depth_at_bottom, emissions_with_stopped_growth, historical_gdp, im_3_emissions, im_3_emissions_vs_rs, im_3_ff_co2, relative_emissions_to_equity, relative_emissions_to_equity_target, co2eq_gap_closing_percentage, diffusion_flux.

[bpowers]

Final per-cluster disposition after C-LEARN-residual Phases 1-3 + the runlist-determinism fix

The bulk of the residual this umbrella tracked is now reconciled. After Phases 1-3 of the C-LEARN-residual plan plus the init-runlist-determinism fix (e24b0080), simulates_clearn passes the hard 1% gate on 3360 matched variables (vs a 348 floor, ~9.7x) and the live failing set shrank from 32 base variables to 21, all fully attributed under a five-category taxonomy in tests/simulate.rs::EXPECTED_VDF_RESIDUAL, pinned exact by clearn_residual_exactness (re-run deterministically, twice).

Cluster 1 -- SAMPLE UNTIL / INIT-of-:NA: / target-policy

Mostly reconciled. last_active_target_year, time_from_target_to_ultimate_target, target_emissions_for_rate, ultimate_target_value_from_rate, depth_at_bottom, and emissions_with_stopped_growth all dropped out of the failure set (passthrough-INIT handling in Phase 3, plus the init-runlist-determinism fix e24b0080 that stopped depth_at_bottom's per-layer init values from permuting).

Remaining (boundary category, genuine documented remainder):

• last_set_target_year -- every cell is -2*NA: Vensim computes -1.298e33, Simlin keeps the bare :NA: sentinel -6.49e32 (crate::float::NA).
• historical_gdp -- non-:NA: cells match exactly (e.g. [oecd_us] 6.667e4); only its -2*NA cells diverge, same as above.

:NA:-arithmetic is confirmed correct and explicitly out of scope (see float::NA): the sentinel is finite by design so the IF THEN ELSE(x = :NA:, ...) existence idiom works. These two stay carved out as boundary with a sourced reason.

Cluster 2 -- numeric tail

The substantive im_3_* / relative_emissions_* chain (rel ~0.247 on meaningful values) is reconciled -- im_3_emissions, im_3_emissions_vs_rs, im_3_ff_co2, relative_emissions_to_equity, relative_emissions_to_equity_target all dropped out of the failure set after the Phase 1-3 engine fixes. There is no remaining engine-genuine numeric divergence.

Remaining (benign-near-zero category):

• co2eq_gap_closing_percentage -- a ratio of near-equal small values, peak ~1.1e-2 (vdf -6.8e-4 vs sim 6.9e-4); divergence only on near-zero magnitudes.
• diffusion_flux -- ~2% on a small early transient (~7% of cells, vdf 0 vs sim ~5e-10), matching late; cross-simulator near-zero noise.

Cluster 3 -- NaN-vs-:NA: representation gap

Not in the failure set, by construction. Where Vensim writes literal IEEE NaN and Simlin writes the finite :NA: sentinel (e.g. slr_inches_from_2000), the comparator NaN-skips those cells and the finite tail matches within tolerance, so the series is neither all-NaN nor has failing cells. This is pinned independently of C-LEARN by the new unit test classify_vdf_ident_nan_vs_na_skips_without_failing. The representation gap itself remains tracked here (not excluded).

The dominant former cluster (the lookup/data 0+0 import) moved to a reader-decode artifact

The 17 graphical-function / lookup-only bases are now classified as VDF-decode-artifact (engine proven correct; the Ref.vdf reader mis-decodes the standalone descriptor columns). The engine 0+0 bug is fixed (#590, Phase 1); the scalar reader half is fixed (d69754bc); the arrayed half + 4 scalar no-distinct-column cases are tracked in #597.

Net

This umbrella's genuine engine remainder is now just the :NA:-arithmetic boundary (2 bases, out of scope) and the benign-near-zero noise (2 bases). Everything else either reconciled (Phases 1-3 / e24b0080) or moved to the reference-reader-decode bucket (#597). Suggest narrowing this issue's scope to the :NA:-arithmetic representation note (or closing if that is deemed working-as-intended), with #597 carrying the reader work.