buildRustCrate: darwin Rust builds are non-reproducible — crate SVH absorbs the build dir (source path + cwd + build.rs OUT_DIR)

[drzln]

Summary

Darwin Rust builds via buildRustCrate (and substrate's lib/build/rust/lockfile-builder.nix) are not reproducible: the same .drv produces a different crate SVH (Strict Version Hash, in the .rustc metadata) on each build. This is the root cause of the 2026-06-02 rio attic cache-poison incident (E0463/E0460 can't find crate): a darwin-built crate pushed to a shared cache carries a divergent SVH vs. what a consumer was compiled against.

Linux is unaffected only because its build sandbox pins the build directory to /build. Darwin has no such sandbox, so the build runs in a per-build-random $NIX_BUILD_TOP.

Root cause (isolated with controlled rustc runs)

The crate SVH absorbs three build-environment inputs, all derived from the build directory:

1. Absolute source path (rustc SourceMap) → fixed by --remap-path-prefix. (buildRustCrate already passes one; on darwin it does take effect.)
2. Compilation cwd, baked raw into the crate hash separately — --remap-path-prefix does not reach it. Fixed only by -Z remap-cwd-prefix=. (requires RUSTC_BOOTSTRAP=1) or by running rustc in a deterministic cwd.
3. build.rs-baked absolute OUT_DIR ($(pwd)/target/build/<crate>.out, configure-crate.nix) that a crate include!s into its lib (e.g. derive-deftly-macros). No remap reaches this — it's a string value, not a span. Fixed only by a deterministic OUT_DIR ⇒ deterministic build dir.

Critically: cache poison is metadata/SVH non-determinism only — object-code drift is harmless (a consumer resolves a dependency by its metadata/SVH, never its object). So the correct acceptance gate is metadata determinism, not whole-output nix-store --realise --check byte-identity (the latter fails on harmless darwin object drift and masks the real signal).

Minimal reproduction

# trivial crate, identical env, vary only the absolute source path:
rustc --crate-type lib --crate-name x /tmp/a/lib.rs -Cmetadata=fixed --emit=metadata -o m1.rmeta
rustc --crate-type lib --crate-name x /tmp/b/lib.rs -Cmetadata=fixed --emit=metadata -o m2.rmeta
cmp m1.rmeta m2.rmeta            # DIFFER  (source path in metadata)
# + --remap-path-prefix=/tmp/a=/S and =/tmp/b=/S  -> IDENTICAL
# vary only the cwd (relative src), remap source -> still DIFFER
# + RUSTC_BOOTSTRAP=1 -Z remap-cwd-prefix=.        -> IDENTICAL

For (1)+(2) the two flags compose to full metadata determinism across lib / relative-src / --extern-dep / proc-macro shapes. (3) needs a deterministic build dir; remaps cannot reach a build.rs-baked value.

Proposed fix (substrate-side)

Replicate the one thing the linux sandbox provides — a deterministic real build directory on darwin — so source path, cwd, and OUT_DIR are all deterministic at once (this is what makes linux reproducible). Implementation is non-trivial: a naive cp -a to a $out-derived dir regressed in testing (multi-source interaction), and the remap-flag pair leaves a 16-byte OUT_DIR residual. Cleanest long-term: have buildRustCrate run rustc in a deterministic dir and/or set a deterministic OUT_DIR (candidate for an upstream nixpkgs change; -Z remap-cwd-prefix partially mitigates but is unstable + doesn't address OUT_DIR).

Current posture

Worked around at the fleet level — rio (Linux, reproducible) is the sole cache filler; darwin consumes but does not push. Full analysis + the controlled-isolation data: pleme-io/nix:docs/darwin-rust-cache-reproducibility.md. This issue tracks the load-bearing fix (deterministic darwin build dir) so darwin can safely fill again.