Skip to content

test+docs(cms): internal pkg tests, mutation baseline, honest README#17

Merged
jamestexas merged 1 commit into
mainfrom
feat/audit-r5-internal-and-baseline
Jun 17, 2026
Merged

test+docs(cms): internal pkg tests, mutation baseline, honest README#17
jamestexas merged 1 commit into
mainfrom
feat/audit-r5-internal-and-baseline

Conversation

@jamestexas

Copy link
Copy Markdown
Collaborator

Summary

Round 5 of the audit-equivalent hardening β€” closes the largest remaining mutation-testing gap, documents the lived-mutant inventory, and updates the README to reflect the testing methodology honestly.

Stacked on PR #16 β€” when #16 merges, this PR auto-retargets to main.

Why this round, explicitly

After rounds 1–4 we hit 82.19% efficacy with three real bugs fixed. But there were three structural gaps the prior work couldn't address:

  1. pkg/cms/internal was invisible to mutation testing. No test file linked it, so gremlins reported every mutant as NOT COVERED.
  2. No public inventory of what the test suite can't kill. A maintainer or reviewer would have to re-run gremlins and interpret each surviving mutant from scratch.
  3. No public framing of the testing methodology. README claimed "Well-tested" but didn't enumerate what that meant β€” and didn't make clear this is NOT an external audit.

What's added

1. Internal package tests (pkg/cms/internal/asn1util_test.go)

Direct table tests for `MarshalSequenceHeader`, `MarshalSetHeader`, `MarshalImplicitHeader` and `MarshalSafe` β€” same byte-exact boundary-table approach as `pkg/cms/length_boundary_test.go` from round 3, plus `TestConstants` to pin the documented size limits.

Metric Before After
`pkg/cms/internal` efficacy (untested) 100.00%
`pkg/cms/internal` killed 0 18
`pkg/cms/internal` lived 0 0

Side effect: `pkg/cms` itself rises from 82.19% β†’ 83.40% because gremlins now sees additional coverage flowing through the internal helpers when `pkg/cms` tests run.

2. MUTATION_BASELINE.md

An explicit inventory of the 44 lived + 23 not-covered mutants on `pkg/cms`, decomposed into four categories with one-line justifications each:

  • A β€” Logically equivalent (cannot be killed without removing real code, ~15)
  • B β€” Defensive bounds (effectively unreachable at scale, ~12)
  • C β€” Error-message format strings (low security value, ~5)
  • D β€” Genuine boundary cases (addressable, low return, ~10)

Plus promotion criteria for future PRs and an explicit "not independently audited" disclaimer at the top.

3. README "Testing & Hardening" section

Honest framing throughout β€” leads with "not independently audited." Tabulates every test layer with the file that implements it. Lists the four real bugs found and fixed by the methodology with PR links. Documents what the methodology does not cover (cryptographic primitives β†’ stdlib, side-channels β†’ not tested, HSM β†’ out of scope).

Also fixes a stale claim in Limits β€” "SHA-256 only" β†’ SHA-256/384/512 per RFC 8419.

What this PR does not do

  • Does not raise the CI mutation gate (still 80%). The 80% floor is for regression detection, not aspirational targets. Raising it now would discourage refactors that temporarily dip efficacy by 1-2%.
  • Does not claim 100% efficacy on `pkg/cms`. See `MUTATION_BASELINE.md` for why this is intentional β€” half the remaining mutants are unkillable without making the code worse.

Validation

  • Full suite passes under `-race` (make test)
  • `make mutation-test` reports 83.40% on `pkg/cms`, 100% on `pkg/cms/internal`
  • `make govulncheck` clean

Test plan

πŸ€– Generated with Claude Code

Base automatically changed from feat/audit-r4-completeness to main June 17, 2026 18:35
Round 5 of the audit-equivalent hardening: close the largest remaining
mutation-testing coverage gap (the internal/ package was invisible to
gremlins because no test file linked it), document the lived-mutant
inventory, and update the README to reflect the testing methodology
without overstating it as an external audit.

What's added:

1. pkg/cms/internal/asn1util_test.go

   Direct table tests for MarshalSequenceHeader, MarshalSetHeader,
   MarshalImplicitHeader and MarshalSafe β€” parallel to the
   length_boundary_test.go work on the sister helpers in pkg/cms.
   Achieves 100% mutation efficacy on the internal/ package (18
   killed, 0 lived). Side-effect: mutation efficacy on pkg/cms itself
   rises from 82.19% to 83.40% because gremlins now sees more
   coverage flowing through the internal helpers when pkg/cms tests
   run.

2. MUTATION_BASELINE.md

   Inventory of the 44 lived + 23 not-covered mutants on pkg/cms,
   categorised:
     A β€” Logically equivalent (no fix possible without weakening code)
     B β€” Defensive bounds (effectively unreachable at scale)
     C β€” Error-message format strings (low security value)
     D β€” Genuine boundary cases (addressable, low return)

   Plus promotion criteria for future PRs and an explicit note that
   the project has not been independently audited.

3. README.md "Testing & Hardening" section

   Honest framing: every layer of testing the repo ships, mapped to
   the files that implement it. Lists the bugs found and fixed by the
   methodology with PR links. Explicit "not independently audited"
   header. Also fixes a stale claim in Limits ("SHA-256 only" β†’ all
   of SHA-256/384/512 per RFC 8419).

Why pkg/cms/internal mutation jumped to 100%:

   The internal package's helpers had no tests linking them, so
   gremlins reported every mutant as NOT COVERED. The new test file
   provides table-driven byte-exact coverage at every length-encoding
   boundary, killing every covered mutant.

Validation: full suite passes under -race; mutation efficacy now
83.40% on pkg/cms, 100% on pkg/cms/internal, CI gate of 80% stays
comfortably below the new floor.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@jamestexas jamestexas force-pushed the feat/audit-r5-internal-and-baseline branch from 1f2f754 to a5310a7 Compare June 17, 2026 18:36
@jamestexas jamestexas merged commit bd0b97f into main Jun 17, 2026
8 checks passed
@jamestexas jamestexas deleted the feat/audit-r5-internal-and-baseline branch June 17, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant