feat(evolution): APT-pattern safety gates with auto-rollback (ANGA-900)#76
feat(evolution): APT-pattern safety gates with auto-rollback (ANGA-900)#76anhermon wants to merge 1 commit into
Conversation
…A-900) Implements SafeApplier with three-stage safety gates: - Pre-hook: backs up current prompt before applying - Apply: writes new prompt to config.toml - Post-hook: runs benchmarks and auto-rolls back if performance regresses >10% Includes: - SafeApplier trait implementation with backup/restore logic - Benchmark runner script (build/test/clippy/fmt metrics) - Auto-rollback on >10% performance regression - Backup management (keeps last 10 backups) - Unit tests for backup and benchmark parsing - Documentation in SAFETY_GATES.md The default_engine() now uses SafeApplier instead of DefaultApplier. Added unsafe_engine() for testing without safety gates. Co-Authored-By: Paperclip <noreply@paperclip.ing>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ae81284dec
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let output = Command::new(&self.benchmark_script) | ||
| .output() |
There was a problem hiding this comment.
run_benchmarks() invokes ./scripts/run-evolution-benchmarks.sh directly, but this commit adds that script with mode 100644 (not executable). On Unix, Command::new(...) returns a permission error, and apply() treats that as a benchmark failure, rolls back, and returns Err, so evolved prompts cannot be applied successfully. Either commit the script as executable (100755) or execute it via bash.
Useful? React with 👍 / 👎.
| passed=$(grep -oP '\d+(?= passed)' /tmp/test-output.txt | tail -1 || echo "0") | ||
| failed=$(grep -oP '\d+(?= failed)' /tmp/test-output.txt | tail -1 || echo "0") |
There was a problem hiding this comment.
The pass-rate calculation parses cargo test --workspace output with grep ... | tail -1, which only reads the final test-summary line. Workspace runs emit multiple summaries (one per test binary/package), so earlier failures can be ignored if the last summary is clean, producing an inflated test_pass_rate and potentially missing regressions that should trigger rollback.
Useful? React with 👍 / 👎.
1 participant
Summary
Implements Safety Gates for Anvil's 5-gate self-evolution engine using the APT (Agentic Prompt Template) pattern. Prevents performance degradation through automatic benchmark validation and rollback.
This addresses ANGA-900 from the APT pattern brainstorm (ANGA-865, Priority 1).
Changes
SafeApplier: New
Appliertrait implementation with three safety gates:~/.paperclip/harness/config.tomlBenchmark runner: Shell script (
scripts/run-evolution-benchmarks.sh) that measures:Backup management: Timestamped backups in
~/.paperclip/harness/prompt-backups/, keeps last 10Integration:
default_engine()now usesSafeApplierinstead ofDefaultApplierDocumentation: Added
crates/evolution/SAFETY_GATES.mdwith full specFiles Changed
crates/evolution/src/safe_applier.rs(new, 430 lines) — APT safety gates implementationcrates/evolution/src/defaults.rs— Updateddefault_engine()to use SafeAppliercrates/evolution/src/lib.rs— Export SafeApplierscripts/run-evolution-benchmarks.sh(new, 86 lines) — Benchmark runnercrates/evolution/SAFETY_GATES.md(new, 178 lines) — DocumentationTest Plan
Verification (from ANGA-900)
Related
/review-selfcommand (APT pattern reference)🤖 Generated with Claude Code