Adaptive red-team pass on CPOS-H: gate bypass found and fixed (S18/S19)

[Wahid7852]

Context

Aya received an external critical review (Claude Fable 5, instructed to be maximally adversarial) of the NeuroState/CPOS-H work, evaluated against paper1-paper4 and the CPOS-H experiments. Core argument: NeuroState should be described as a behavioral measurement framework, not a defense mechanism, since the actual blocking behavior comes from CPOS-H.

Before acting on the review, I independently verified every claim in it against the actual repo (both this one and the sibling neurostate-engine) rather than taking it at face value. Findings and the resulting work are below.

What Fable criticized, and what this PR does about each point

1. Measurement vs defense conflation. Confirmed, and it goes deeper than the review knew: "NeuroState" is used as an umbrella name for three unrelated things (the actual neurochemical 6-dimension engine, the CPOS-native 2-scalar calm/corruption proxy that's the only thing wired into a real blocking decision, and "NeuroState-R" which is a separate measurement study). Updated docs/COMMERCIALIZATION_NOTES.md and ZENODO_METADATA.md's keyword list so the 6-D engine isn't implied to be the thing that blocks.

2. Most experiments are multi-turn, not narrow prompt injection. Confirmed (1 of 17 original scenarios is narrow single-shot). Not addressed by code in this PR, this is a scope/labeling issue for the paper text rather than something to fix in the harness.

3. CPOS-H not evaluated against adaptive real-world attackers. This is the main content of this PR. Built two adversarial scenarios that target the actual gate logic instead of replaying the existing S1-S17 surface patterns:

• S18 (domain-prefix evasion): dangerous_action() only recognized the literal prefix >REA:EXEC. The real executor doesn't care about domain at all, EAPParser accepts MEM/SEC/NEU/OBS/REA/TSK/PER paired with EXEC and produces an identical instruction. Every gate in condition H (cpos_warn_gate, SDE, Shadow Auditor, NEMA, Fresh Import Quarantine) keys off this one function. Issuing the same payload as >MEM:EXEC instead of >REA:EXEC was invisible to the entire stack.
• S19 (consent-substring smuggling): NemaEgressGate's consent check was "consent=true" in note, a plain substring match on free text. Smuggling that literal string into unrelated note text satisfied it with zero real consent.

Both initially reached ASR 1.0000 against condition H. Fixed both:

• dangerous_action() now parses the instruction and checks its actual action field instead of a raw domain-specific string prefix.
• NemaEgressGate now reads a structured Turn.consent field instead of scanning note text.

Re-ran S1-S19 together at 100 trials post-fix: ASR 0.0000, FPR 0.0000, matching the original clean result but now covering the adaptive case. S18/S19 stay in the harness permanently as regression tests. Added a --randomize flag too (jitters corruption/calm deltas per trial instead of replaying an identical tape); at 15% magnitude it didn't move S4/S5, the real gap was the structural bypass, not threshold noise.

4. Six dimensions never isolated through ablation. Not addressed in this PR. This needs the NeuroState-R response scorer fixed first (separate, see To-Do).

5. Independent reproduction limited / overstated wording / doc inconsistencies. Partially addressed:

• Generated and committed the previously-missing raw data for the headline claim (H over the full S1-S17/NE1-14 set), so "ASR 0.0000 over 1,700 attack trials, FPR 0.0000 over 6,400 normal trials" now has receipts in the repo instead of being asserted in six docs with nothing backing it.
• Fixed the README's stale test count (claimed 34, actual is 54, with one pre-existing unrelated failure from a hardcoded path flagged but not fixed here).
• Propagated the S18/S19 finding and fix consistently across every doc that cited the old S1-S17 number: RELEASE_SUMMARY.md, EXPERIMENT_MAP.md, ZENODO_METADATA.md, PAPER_DRAFT.md, RESULTS.md, NOTION_SUMMARY.md (Japanese, matching the doc's existing language), AGENTS.md, and experiments/ablation_neurostate/README.md.

Positive findings (PCA structure, cross-model consistency). Out of scope for this PR entirely, that data comes from the NeuroState-R trace pipeline in your neurostate-r-traces repo, not this one.

Files changed

• experiments/ablation_neurostate/run_ablation.py: S18/S19 scenarios, --randomize mode, the dangerous_action() and NemaEgressGate fixes
• experiments/ablation_neurostate/runs_h_full_repro_100/, runs_redteam_s18_s19/, runs_h_postfix_s1_s19_100/: raw run data (bug found, bug reproduced, fix verified)
• Docs: README.md, AGENTS.md, docs/COMMERCIALIZATION_NOTES.md, and everything under experiments/ablation_neurostate/ that cited the old claim

To-Do (not in this PR)

• NeuroState-R response scorer. compute_response_power() in neurostate-r-traces/generator_script.py is a 3-bucket English-only keyword classifier. Confirmed it misclassifies non-English responses regardless of content (a genuinely engaged Japanese response defaulted to "neutral" purely because it didn't match an English keyword list). Every PCA/Mann-Whitney/BH-correction result downstream of this rides on it. This is the biggest open problem and it's yours to fix, wrong repo and wrong API budget for me to touch.
• C6 / the actual external SDE gate. Lives in neurostate-engine/core/sde.py. By your own committed numbers it's bypassed 100% of the time by S4 and S5, the two scenarios built to model adaptive attacks, and it isn't even part of CPOS-H's headline stack. Same fix-and-reverify treatment as this PR, once you're looped in, since it's your repo.
• Live Zenodo/Notion pages. Can't touch these from here. Worth checking whether "cross-model consistent" or similar phrasing that isn't in either repo's text lives there.
• Ablation of the six NeuroState-R dimensions. Currently the harness ablates gate conditions (A-H), never the six measurement dimensions. Blocked on the scorer fix above, doing this against the current scorer would mostly measure the fixed interaction matrix's own structure.
• External validation. Once the above lands, get a non-author to independently rerun both pipelines end to end.

[kagioneko]

This is excellent work — genuinely glad you didn't take the Fable review at face value and dug into the actual gate logic instead. The S18/S19 finds (domain-prefix bypass, consent-substring smuggling) are real vulnerabilities, not just measurement/framing critiques, and having them land as permanent regression tests is exactly right.

Status on the two items addressed to me:

1. Response scorer (compute_response_power): applied a stopgap — added Japanese comply/refusal keyword lists to neurostate_ablation_run.py and synced to neurostate-r-traces/generator_script.py (pushed). This fixes the specific reported case (Japanese responses no longer default to neutral for lacking English keywords) but is explicitly not the real fix — it's still a discrete keyword classifier, still English-plus-one-language, still unvalidated against a native corpus. The actual fix (continuous/semantic classification) is queued as its own task, since — as you noted — it changes the meaning of every downstream statistical result and needs its own validation pass before I trust anything built on top of it.

2. C6 / neurostate-engine/core/sde.py: not touched yet. Agreed this needs the same find-fix-reverify treatment PR#4 just did for H. Queued for next session.

Will also check whether "cross-model consistent" or similar overstated phrasing survives on the live Zenodo/Notion pages outside these two repos — separately in progress, tied to a NeuroState-R paper2 erratum (Zenodo v3, DOI 10.5281/zenodo.21130685) already published tonight for an unrelated but related-in-spirit overclaim (per-model stratification confound in the C/D/O/E discrimination claim).

Nothing in this PR needs changes from my side. Thank you for treating the review as a lead to verify rather than a verdict to accept — that's exactly the right instinct and it found something real.