Skip to content

Harden Anthropic prefix cache stability (no prior-turn mutations)#102

Closed
JerrettDavis wants to merge 15 commits intochopratejas:mainfrom
JerrettDavis:fix/anthropic-prefix-cache-stability
Closed

Harden Anthropic prefix cache stability (no prior-turn mutations)#102
JerrettDavis wants to merge 15 commits intochopratejas:mainfrom
JerrettDavis:fix/anthropic-prefix-cache-stability

Conversation

@JerrettDavis
Copy link
Copy Markdown
Contributor

Summary

  • Harden Anthropic prompt-prefix stability so prior turns are never mutated.
  • Enforce deterministic tool ordering across Anthropic request, batch, and CCR continuation paths.
  • Restrict dynamic adjustments (memory/proactive context/image transforms) to the final turn only.
  • Cap token-headroom freeze behavior by provider cache state and strict prior-turn freeze policy.
  • Fix uv run dependency resolution for Python 3.10 by marker-gating anyllm extra to Python 3.11+.

Key Changes

  • Added deterministic tool canonicalization in Anthropic proxy handler and batch flow.
  • Added strict freeze policy: all previous turns frozen; only final turn mutable.
  • Disabled CCR system-instruction injection when prefix is frozen.
  • Made memory/proactive context append only to final user turn (never system/prior turns).
  • Applied deterministic tool sorting in Anthropic CCR batch continuation calls.
  • Added/expanded Anthropic cache-stability regression tests.

Tests

  • uv run --extra dev pytest tests/test_proxy_anthropic_cache_stability.py -q (11 passed)
  • uv run --extra dev pytest tests/test_ccr_batch_processor.py tests/test_proxy_hooks_regression.py tests/test_compression_cache.py tests/test_cache/test_prefix_tracker.py -q (99 passed)

@JerrettDavis JerrettDavis marked this pull request as draft April 4, 2026 19:01
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 4, 2026

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 56.04027% with 131 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
headroom/proxy/handlers/anthropic.py 50.95% 77 Missing and 26 partials ⚠️
headroom/proxy/handlers/openai.py 40.62% 18 Missing and 1 partial ⚠️
headroom/cache/prefix_tracker.py 77.77% 2 Missing ⚠️
headroom/ccr/batch_processor.py 77.77% 2 Missing ⚠️
headroom/proxy/modes.py 90.47% 1 Missing and 1 partial ⚠️
headroom/ccr/mcp_server.py 0.00% 1 Missing ⚠️
headroom/proxy/cost.py 50.00% 0 Missing and 1 partial ⚠️
headroom/proxy/server.py 90.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@JerrettDavis JerrettDavis deleted the fix/anthropic-prefix-cache-stability branch April 11, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants