Show & Tell: How I caught 287 AI rule violations across 50 repos with SpecLock

[sgroy10]

Show & Tell: How I caught 287 AI rule violations across 50 repos with SpecLock

I ran SpecLock's semantic audit across 50 active repos in my workspace and what I found was... humbling.

Total violations caught: 287 across repos where I thought my CLAUDE.md rules were being respected by Claude, Cursor, and Copilot. They weren't.

Full writeup with charts and repo-by-repo breakdown here: https://sgroy10.github.io/speclock/blog/50-repos-claude-md-violations.html

The 7 drift patterns I saw repeatedly

1. The "cleanup" drift — Rules like "never delete test fixtures" got violated when the AI interpreted "clean up unused files" as a green light to purge things marked as critical.
2. The synonym gap — CLAUDE.md said "don't remove the auth middleware." AI "refactored it out." Same outcome, different word.
3. The scope creep drift — User asked for a 1-line fix. AI rewrote the whole module. CLAUDE.md said "minimal diffs only." Ignored.
4. The silent import drift — Rule: "no new dependencies without approval." AI added 4 npm packages in a single PR.
5. The architectural drift — CLAUDE.md pinned a specific pattern (e.g., "use repository pattern for DB access"). AI quietly introduced direct ORM calls.
6. The test deletion drift — Tests that were "flaky" got deleted instead of fixed. Rule said "never delete tests without human review."
7. The config drift — Env vars, feature flags, and tsconfig settings got "optimized" away despite being locked.

Why keyword matching misses these

Most naive hooks do grep -i "don't delete" and call it a day. That catches maybe 30% of violations. The rest hide behind synonyms, rewordings, and indirect actions. SpecLock's semantic engine uses a synonym taxonomy + intent detection to catch the other 70%.

Your turn

Have you run a semantic audit on your repos yet? Drop the numbers in a reply:

npx speclock audit --semantic --all

How many violations did you find? Which drift pattern hit you hardest? Would love to crowdsource a drift pattern atlas from this community.

Related reading:

• SpecLock FAQ
• How SpecLock catches violations keyword matching misses