fix: strip billing header from Anthropic system prompt for prefix cache

[janhilgard]

Summary

• Strip x-anthropic-billing-header from system prompt before tokenization to enable prefix cache reuse across turn boundaries

Problem

Claude Code injects a billing/tracking header into the Anthropic Messages API system prompt:

x-anthropic-billing-header: cc_version=2.1.100.7b4; cc_entrypoint=cli; cch=eb6c6;
You are Claude Code...

The cch= hash changes with every request, causing token sequences to diverge at position ~40. This completely defeats prefix cache — every request requires full prefill of 60K+ tokens (~50 seconds on Gemma 4 26B-A4B).

Fix

One-line regex strip in anthropic_adapter.py before the system text is passed to the chat template:

system_text = re.sub(r"x-anthropic-billing-header:[^\n]*\n?", "", system_text)

Result

Metric	Before	After
Prefix match	40/60K tokens (0.07%)	60K/60K tokens (99.9%)
Time per request	50s	3.65s
Throughput	1.9 tok/s	42.1 tok/s
Speedup	—	13.7x

Test plan

• Consecutive Claude Code Anthropic requests match 60K/60K tokens (prefix hit)
• First request (cold cache): full prefill ~49s
• Second request (warm cache): 3.65s (only 34 new tokens to process)
• Non-Anthropic (OpenAI) requests unaffected
• Anthropic requests without billing header unaffected

🤖 Generated with Claude Code