Skip to content

Support system prompt caching via system_prompt_cache_type provider option#410

Open
GioChocolateBro wants to merge 1 commit intolaravel:0.xfrom
GioChocolateBro:fix/system-prompt-caching-v2
Open

Support system prompt caching via system_prompt_cache_type provider option#410
GioChocolateBro wants to merge 1 commit intolaravel:0.xfrom
GioChocolateBro:fix/system-prompt-caching-v2

Conversation

@GioChocolateBro
Copy link
Copy Markdown

Fixes #119

Problem

Anthropic's prompt caching can dramatically reduce input token costs for agents with large system prompts. The AnthropicGateway passes system prompts as plain strings — there's no way to attach the cache_control metadata that Anthropic requires for caching.

Solution

Add a system_prompt_cache_type provider option. When set, BuildsTextRequests formats the system prompt as a cached content block before sending to the API.

This is 12 lines in one trait. When the option is not set, behavior is completely unchanged — the isset check fails and the system prompt stays a plain string.

Usage

Agents opt in via HasProviderOptions:

use Laravel\Ai\Contracts\HasProviderOptions;
use Laravel\Ai\Enums\Lab;

class MyAgent implements Agent, HasProviderOptions
{
    public function providerOptions(Lab|string $provider): array
    {
        return match ($provider) {
            Lab::Anthropic, 'anthropic' => [
                'system_prompt_cache_type' => 'ephemeral',
            ],
            default => [],
        };
    }
}

How it works

In BuildsTextRequests::buildTextRequestBody(), after provider options are resolved:

  1. Check if system_prompt_cache_type is set
  2. Convert $body['system'] from a plain string to an array of content blocks with cache_control
  3. unset the key from $providerOptions so it doesn't leak into the API body via array_merge

Both formats (string and content block array) are valid per the Anthropic Messages API.

Why not strip cache_control from provider options (PR #365)?

That approach was closed because it would break the existing tool_result_cache_type mechanism. This PR doesn't touch provider options flow at all — it only consumes and removes its own key.

Tests

3 new tests in ProviderOptionsTest.php:

  • system_prompt_cache_type formats system prompt as cached content block
  • system_prompt_cache_type is not included as a top level key in request body
  • system prompt remains a string when agent does not set system_prompt_cache_type

Results

Tested in production on a multi-step analytics agent with a ~8.5k token system prompt:

Full-price tokens Cache write Cache read
First request (cold) 331 8,590 0
Subsequent requests 331 0 8,590

~90% reduction in effective input token cost. Cache stays warm across conversations within the 5-minute TTL.

Changes

  • BuildsTextRequests.php — 12 lines added
  • CachedSystemPromptAgent.php — test fixture
  • ProviderOptionsTest.php — 3 new tests
  • No interface changes, no breaking changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Anthropic prompt caching (cache_control on system messages)

1 participant