[CI debug - DO NOT MERGE] Diagnose ninja cascade-rebuild on test side#195
Closed
lamb-j wants to merge 1 commit into
Closed
[CI debug - DO NOT MERGE] Diagnose ninja cascade-rebuild on test side#195lamb-j wants to merge 1 commit into
lamb-j wants to merge 1 commit into
Conversation
Codegen and Comgr test jobs cascade-rebuild even with tar -xmf. Add
diagnostic steps after untar in both jobs to capture:
- mtime + size of build/.ninja_deps, .ninja_log, build.ninja
(corruption or absence would explain "ninja thinks everything is
dirty")
- mtime of source CMakeLists.txt (for comparison vs build.ninja
mtime — settles the "is mtime the trigger" question)
- ninja -d explain -n output, which prints why ninja decides to
rebuild each target without actually building (this is the gold
signal — tells us if it's mtime, command-hash, depfile-missing,
or something else)
One-shot debug PR. Once we see the explain output we'll know what to
fix and revert this. Do not merge.
Contributor
🔴 New failures (0) — likely caused by this PR(none) 🟢 Fixed by this PR (0) — failing on baseline, passing here(none)
|
This was referenced May 10, 2026
lamb-j
added a commit
that referenced
this pull request
May 11, 2026
Diagnostic in #195 showed test-side ninja invocations cascade-rebuild because tar -m sets per-file mtimes from sequential extraction order, leaving build.ninja older than CMakeCache.txt within the same build dir: ninja explain: output build.ninja older than most recent input CMakeCache.txt (1778450813199019645 vs 1778450820955083819) That triggers ninja's auto-regen rule, cmake regenerates build.ninja with subtly different command lines, and ninja's command-hash check then rebuilds every .o it has on file. Fix: explicitly touch build.ninja after untar so it's the newest file in the build tree. Trivial 3-file touch, applies to all 3 test jobs via the shared "Untar build trees" step pattern.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After #194 landed, codegen and Comgr test jobs cascade-rebuild despite `tar -xmf`. Codegen rebuilds 2926 targets (~10 min); Comgr rebuilds 228 targets (~4 min, fits in budget so it appears green).
Adds diagnostic steps after the untar in both jobs to capture:
Once the diagnostic comes back we'll know which of these is the cause:
Then we land the targeted fix in a separate PR and revert this debug.