Skip to content

fix(check): fall back to tail for empty TAP failure summaries#127

Draft
YforC wants to merge 1 commit into
amarbel-llc:masterfrom
YforC:fix-empty-tap-failure-tail
Draft

fix(check): fall back to tail for empty TAP failure summaries#127
YforC wants to merge 1 commit into
amarbel-llc:masterfrom
YforC:fix-empty-tap-failure-tail

Conversation

@YforC

@YforC YforC commented Jun 9, 2026

Copy link
Copy Markdown

Why

When a tap-ndjson hook exited non-zero but all parsed records were SKIP or TODO, buildFailureSummary returned an empty string. runHookCompact still emitted failure:, which hid the raw hook output tail and gave agents no useful failure context.

Fixes #86.

What Changed

The hook failure path now only emits failure when buildFailureSummary returns a non-empty summary. Otherwise it falls back to tail, preserving the raw hook output. A regression test covers a non-zero hook whose TAP records are all SKIP/TODO.

Verification

  • go test ./internal/check -run ''^(TestBuildFailureSummary|TestRunHookCompactShape_TapNDJSONSkipTodoFallback)$'' -count=1 -v

Full go test ./internal/check -count=1 is blocked on this Windows machine because existing hook tests require sh on PATH.

@friedenberg

Copy link
Copy Markdown
Contributor

Thanks for this fix — its intent just landed via commit 67e28ef (branch bright-cedar, merging to master shortly), ported rather than merged directly: the FDR 0015 re-platform (merge/check now emit ndjson-crap instead of TAP) rewrote runHookCompactContext into runHookPhase, so the diff here no longer applies. The ported version implements the same fallback — empty failure summary → ring tail — at the new single selection point, which also covers the ndjson-crap format's identical empty-summary case, with regression tests for both formats (TestRunHookPhaseShape_TapNDJSONAllSkipFallsBackToTail / _NdjsonCrapAllSkipFallsBackToTail, modeled on this PR's all-SKIP/TODO repro). #86 will auto-close when the commit reaches master.

This PR can be closed as superseded once that lands — leaving that to you rather than closing someone else's PR.

:clown: Clown 0.4.0+cd38c2d (amarbel-llc/clown@cd38c2d)

friedenberg added a commit that referenced this pull request Jun 10, 2026
A structured-format hook (tap-ndjson or ndjson-crap) that exits non-zero
with only SKIP/TODO records yields an empty failure summary, leaving the
failing test record's output diagnostic blank and hiding the raw hook
tail. Fall back to the ring tail whenever the summary is empty.

Ports PR #127 by @YforC, which fixed this for the TAP-era runHookCompact
before the FDR 0015 re-platform rewrote the emission path; the fallback
now also covers the ndjson-crap analogue.

:clown: Clown 0.4.0+cd38c2d (amarbel-llc/clown@cd38c2d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

runHookCompact emits empty failure: when parsed records are all SKIP/TODO

2 participants