core(tdr): add Contract Test Hygiene section#19
Merged
Conversation
Ratified acceptance scenarios must not be silently skipped via skipUnless / --gtest_filter / equivalents. Pending-implementation scenarios use the runner's explicit expected-fail surface; source-shape proxies may detect implementation arrival but must not gate assertions. Drives the Path A tombstone pattern toward an explicit CI state distinguishable from skipped/passing/failing — removing an expected-fail marker becomes a reviewable diff event rather than a silent decorator drop coupled to unrelated work. Source amendment package and pilot consumers (cms-pm/cockpit chunk-8.0t): - docs/governance/amendments/2026-05-04-test-hygiene-tdr-amendment-package.md - tests/_contract_support.py - docs/validation/phase-8/predicates/scn-8.0t.json Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This was referenced May 5, 2026
cms-pm
added a commit
that referenced
this pull request
May 6, 2026
New optional tooling adapter for projects with substantial HiL or SiL
evidence capture. Routes deterministic capture-and-compare work off
the architect-tier agent onto a worker-tier agent driven by a pinned
predicate.json.
Routing contract:
- architect (Opus or equivalent) authors predicate.json at chunk-plan
time; worker (Haiku or equivalent) runs CLI, diffs against predicate,
emits {scn, status, evidence_uri, predicate_hash} JSONL
- worker MUST NOT load architecture context or mutate predicate
- architect re-engages only on non-empty diff or unexpected status
Compatible with the contract test hygiene rule (PR #19) and the
scenario status vocabulary (PR #20). Compounds with the RTK tooling
adapter on shell-visible runner invocations.
Pilot consumer: cms-pm/cockpit Phase 8
- docs/validation/phase-8/predicates/scn-8.0t.json
- scripts/validation/run_phase8_0t_fast_queue_backpressure.py
Predicate schema is project-local at 0.1.0-pilot; will be promoted
to a ratified contract under contracts/ after a second project adopts.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a normative Contract Test Hygiene section to
core/AI_ASSISTED_TDR_METHODOLOGY.md:skipUnless/--gtest_filter/ equivalentsexpectedFailure,DISABLED_,xfail)Motivation
Path A tombstone scenarios in consumer projects (cms-pm/cockpit Phase 8) were using
@unittest.skipUnless(_proxy_function(), reason)to silently skip tests until implementation lands. This produces three failure modes that are invisible in CI:False, so the test stays skipped — drift goes undetected.expectedFailuremakes "not yet implemented" an explicit CI state. Removing the decorator becomes a deliberate, diff-reviewable act when implementation arrives, and a wrong-symbol implementation goes red loudly.This pattern is the Python analog of GoogleTest's
DISABLED_prefix and pytest'sxfail, which the Avoid Fragile Tests paper cites as the right tool for tracked-but-not-yet-passing scenarios.Pilot consumers (cms-pm/cockpit, chunk-8.0t)
The pilot artifacts are already landed in cms-pm/cockpit on
chunk-8.0t-status-additive-and-behavioral:docs/governance/amendments/2026-05-04-test-hygiene-tdr-amendment-package.mdtests/_contract_support.pydocs/validation/phase-8/predicates/scn-8.0t.jsontests/test_phase8_0t_sideband_fast_queue_backpressure_contract.pyThe 8.0t contract suite (6/6 green) was the first chunk authored under this rule — SCN-8.0t-03 and SCN-8.0t-05/-06 run unconditionally and produce real pass/fail.
Stacked PRs (one item per commit, one item per PR)
This is PR 1 of 4 in the test-hygiene amendment package:
tdr/contract-test-hygieneevidence/scenario-status-vocabularyadapters/hil-sil-predicate-routerastaire/scenario-ledger-collectionTest plan
core/EVIDENCE_CONTRACT.mdonce PR ADG-BOOTSTRAP-01: graphify consumer scripts resolve the wrong root #2 lands🤖 Generated with Claude Code