build: doc-path audit — verify every prose file-ref points at a real …#57
Merged
Conversation
…file # Why The bidirectional spec-coverage audit handles drift for tests ↔ specs. What it doesn't handle is drift for prose ↔ files: a file gets renamed, a doc gets moved to docs/archive/, a helper changes location — and CLAUDE.md / README.md / skills.md / TRIAGE.md / docs/*.md keep referencing the old paths long after the link is dead. PR #55 (docs-consolidation) made this concrete: archiving 3 historical files could have silently broken any cross-doc reference to them. The gate would not have caught it. # What Adds `scripts/check-doc-paths.sh` — a pure-bash audit: - Walks CLAUDE.md, README.md, skills.md, TRIAGE.md, TESTS.md, and every current docs/*.md (the archive dir + test-review snapshots are explicitly excluded — they're time-bound). - Extracts two kinds of file references: 1. Markdown-link targets: `[text](./path)` / `[text](path)` 2. Backtick literals that look like paths (contain `/` AND end in a known extension or `/`) - Resolves each candidate first relative to the doc, then relative to the repo root (the prose convention in docs/ is repo-root-relative). - Ignores: URLs, glob/brace patterns, placeholder interpolation, HTTP/API endpoint paths (`/api/users/me`, `/oauth2/sign_out`, `/god-mode/`, ...), generated artefacts (`playwright-report/`, `node_modules/`), and a small documented allowlist for cross-repo refs (foss-server-bundle/) + not-yet-built files (`.github/workflows/e2e-prod.yml` — referenced in README but the production workflow hasn't shipped). - Returns non-zero if any ref points at a missing file. Output lists each break as `<file>:<line>: missing → <raw_path>`. Wired into: - `make audit-paths` (callable on its own) - `make pre-commit` (alongside typecheck + spec-coverage audit) - `.github/workflows/doc-path-audit.yml` (runs on PRs that touch any *.md, the script, or its workflow; runs on push to main) # Fixed by this run The audit immediately surfaced 3 real broken refs in the existing docs: - CLAUDE.md:116 — `app-rules/RULES.md` → correct path is `vendor/openspec/skills/app-rules/RULES.md` - README.md:292 — `security/` → should be `tests/security/` - TRIAGE.md:41 — `security/headers.spec.ts` → should be `tests/security/headers.spec.ts` Each found and fixed in this commit. # Counts Final audit run: 270 references checked across 7 files, 0 missing. Pre-commit gate green (typecheck + spec-coverage + doc-path). # Out of scope (follow-up notes) - `.github/workflows/e2e-prod.yml` is referenced in README but not yet built. Added to IGNORED_PATTERNS with a TODO comment; remove the ignore once the production workflow ships. - `docs/test-review-*.md` and `docs/archive/*` are excluded by design (time-bound snapshots). If we want them audited too, revisit the exclusion list.
CI surfaced one false positive after merging: CLAUDE.md line 21
contains the backtick literal `git log -- tests/`. The path-detection
heuristic was "contains a slash AND ends in .ext or /" — `tests/`
matched that even though the full literal is a shell command, not a
file path.
Tightening the heuristic to also require no whitespace inside the
backtick literal catches this class without losing real coverage:
file paths in prose never contain spaces (they're either single
tokens like `tests/auth/foo.spec.ts` or markdown links). Shell
commands, prose phrases ("`git log -- tests/`", "`cd tests/`"), and
sentence fragments all do contain spaces.
Verified against the CI-surfaced case + the two skills.md references
to docs/spec-review-checklist.md (which now exist on origin/main
after PR #56 merged but not yet on this branch's local main).
TESTS.md was already flagged as "hand-maintained, may lag — git log is the truth" in CLAUDE.md after PR #55. That's a soft deprecation; this commit follows through. The catalog's content is redundant with: - @SPEC tags above each test - the audit table at docs/spec-coverage.md - per-file head comments Drift between TESTS.md and reality has been the highest of any doc in the suite (every test add/remove needs a hand-edit). Retiring it removes the maintenance burden without losing information. Changes: - TESTS.md → docs/archive/TESTS.md (preserved for provenance) - docs/test-review-2026-05-{22,30}.md → docs/archive/ (the 2 review snapshots from prior sessions that were sitting untracked at the repo root level of docs/; same time-bound shape as the test-review-2026-05-20 that PR #55 already archived) - CLAUDE.md "Read these in order": point at file-head comments + `grep -nA1 "// @SPEC" tests/` instead of TESTS.md - README.md opening highlight: point at docs/spec-coverage.md (which has the requirement-by-requirement table) instead of TESTS.md (which had the per-test catalog by another name) - scripts/check-doc-paths.sh: drop TESTS.md from DOC_FILES (no longer at the repo root) After this: - Live doc set at root: CLAUDE.md, README.md, skills.md, TRIAGE.md (4, down from 5) - Archive: 4 historical files + the new TESTS.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…file
Why
The bidirectional spec-coverage audit handles drift for tests ↔ specs. What it doesn't handle is drift for prose ↔ files: a file gets renamed, a doc gets moved to docs/archive/, a helper changes location — and CLAUDE.md / README.md / skills.md / TRIAGE.md / docs/*.md keep referencing the old paths long after the link is dead.
PR #55 (docs-consolidation) made this concrete: archiving 3 historical files could have silently broken any cross-doc reference to them. The gate would not have caught it.
What
Adds
scripts/check-doc-paths.sh— a pure-bash audit:[text](./path)/[text](path)/AND end in a known extension or/)/api/users/me,/oauth2/sign_out,/god-mode/, ...), generated artefacts (playwright-report/,node_modules/), and a small documented allowlist for cross-repo refs (foss-server-bundle/) + not-yet-built files (.github/workflows/e2e-prod.yml— referenced in README but the production workflow hasn't shipped).<file>:<line>: missing → <raw_path>.Wired into:
make audit-paths(callable on its own)make pre-commit(alongside typecheck + spec-coverage audit).github/workflows/doc-path-audit.yml(runs on PRs that touch any *.md, the script, or its workflow; runs on push to main)Fixed by this run
The audit immediately surfaced 3 real broken refs in the existing docs:
app-rules/RULES.md→ correct path isvendor/openspec/skills/app-rules/RULES.mdsecurity/→ should betests/security/security/headers.spec.ts→ should betests/security/headers.spec.tsEach found and fixed in this commit.
Counts
Final audit run: 270 references checked across 7 files, 0 missing. Pre-commit gate green (typecheck + spec-coverage + doc-path).
Out of scope (follow-up notes)
.github/workflows/e2e-prod.ymlis referenced in README but not yet built. Added to IGNORED_PATTERNS with a TODO comment; remove the ignore once the production workflow ships.docs/test-review-*.mdanddocs/archive/*are excluded by design (time-bound snapshots). If we want them audited too, revisit the exclusion list.Motivation / Background
This Pull Request has been created because:
Detail
This Pull Request changes:
Additional information
TIP: Provide additional information such as screenshots, benchmarks, reference to other repositories or alternative solutions
Checklist
Before submitting the PR make sure the following are checked:
[Fix #issue-number]