test+ci(cms): RFC-traceable tests, named threat suite, mutation gate#16
Merged
Conversation
Round 4 of the audit-equivalent hardening shifts from per-mutant
interventions to *completeness as a property*. Three pieces:
1. RFC-traceable compliance tests (pkg/cms/rfc_compliance_test.go)
Every test is named after the RFC and clause it covers, so an
auditor can answer "what tests does this library have for RFC 5652
§5.3?" with a single grep. Coverage spans:
RFC 5652 §5.1 SignedData.Version whitelist {1, 3, 4, 5}
RFC 5652 §5.3 SignerInfo.Version per SID form (IAS=1, SKI=3)
RFC 5652 §5.3 SignatureAlgorithm must be Ed25519 OID
RFC 5652 §5.4 Required signed attributes: contentType,
messageDigest
RFC 5652 §10.1 DER canonical length encoding (no long-form for
values < 128; no leading zero in long form)
RFC 5652 §11.1 eContentType must be id-data when signedAttrs
are absent (Case 2)
RFC 8419 §3 Ed25519 + SHA-512 mandate; algorithm parameters
MUST be absent (passthrough delegates to existing
load-bearing tests)
RFC 8032 EdDSA deterministic-signature property propagates
through the CMS encoder
Most tests duplicate coverage that exists elsewhere — that's the
point: the spec-mapping IS the contribution. Drift is now visible.
2. Named threat-class tests (pkg/cms/attack_scenarios_test.go)
One test per documented CMS attack class, named for the attack it
prevents. The library's defensive posture is now documented in the
test suite itself, not just implied:
- TestAttack_SignerInfoCrossMessage_Replay
- TestAttack_KeyConfusion_DifferentKey_SameSubject
- TestAttack_NoTrustedRoots_Denied
- TestAttack_TrailingDataInjection
- TestAttack_AlgorithmDowngrade_DigestVsActualBytes
- TestAttack_AttachedEContent_RejectedForDetachedAPI
3. Mutation testing as a CI gate
New 'mutation' job in .github/workflows/ci.yml runs gremlins with
--threshold-efficacy 80 on every PR. The 80% floor reflects the
post-round-4 baseline; surviving mutants past this threshold
indicate test gaps that regress this branch's hardening work. New
PRs that drop efficacy below 80% will fail this gate.
Builder cleanup:
cms_builder_test.go now has SIVersionExplicit / SDVersionExplicit
booleans so tests can force literal-0 versions instead of the "0 =
use default" sentinel. Was needed for the §5.1 v0 rejection test.
Validation:
Full suite (-race) passes at 79.2% statement coverage. Gremlins
reports 82.19% test efficacy unchanged from round 3 — round 4 adds
traceability and gating, not new code-path coverage.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
jamestexas
force-pushed
the
feat/audit-r4-completeness
branch
from
38505d8 to
4de63bd
Compare
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Round 4 of the audit-equivalent hardening shifts framing from "find the next mutant" to completeness as a property.
Stacked on PR #15 — review #15 first; this PR's diff vs main includes round 3 length-boundary tests. The mergeable diff vs #15's branch is the round-4 content only.
What "complete" actually means here
After rounds 1–3, we hit 82% mutation efficacy with three real bugs fixed. But three structural gaps remained:
This PR fills those gaps.
What's added
1. RFC-traceable compliance tests (`pkg/cms/rfc_compliance_test.go`)
Each test is named for the RFC clause it covers. Coverage spans:
Most overlap with existing tests by design — the spec-mapping IS the contribution. If a clause stops being tested, the named test goes red.
2. Named threat-class tests (`pkg/cms/attack_scenarios_test.go`)
3. Mutation testing as a CI gate
New `mutation` job in CI runs `gremlins` with `--threshold-efficacy 80`. PRs that drop test efficacy below 80% fail this gate.
4. Builder cleanup
`cms_builder_test.go` gets `SIVersionExplicit` / `SDVersionExplicit` booleans so tests can force literal-0 versions instead of the "0 = use default" sentinel. Needed for the §5.1 v0 rejection test.
Validation
Mutation efficacy unchanged is expected — round 4 adds traceability and gating, not new code-path coverage. The earlier rounds did the bug-finding; this round documents and locks it in.
Test plan
🤖 Generated with Claude Code