Skip to content

fix(sse): preserve Claude Code cache breakpoints - #8934

Merged
diegosouzapw merged 2 commits into
diegosouzapw:release/v3.8.50from
cryptiklemur:fix/claude-partial-cache-breakpoints
Aug 6, 2026
Merged

diegosouzapw merged 2 commits into
diegosouzapw:release/v3.8.50from
cryptiklemur:fix/claude-partial-cache-breakpoints

Conversation

@cryptiklemur

@cryptiklemur cryptiklemur commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add a cache breakpoint to the latest user message when native Claude Code history has none and the four-breakpoint limit permits it.
  • Preserve existing client message breakpoints unchanged.
  • Keep supported Opus mid-conversation system blocks in messages[].
  • Prevent a mid-conversation cache marker from being hoisted into top-level system, which invalidates the cached conversation prefix.
  • Keep the legacy system-message hoist for unsupported models and providers.

Why this change

Anthropic documents two cache behaviors that apply to the affected Claude Code requests.

First, "Cache writes happen only at your breakpoint." Anthropic checks up to 20 blocks behind each breakpoint for a prior write and supports up to four breakpoints. A request with cache markers on static system or tool content, but none in messages[], does not write a recent conversation-prefix entry. As the conversation grows, the last usable write can fall outside the lookback window. This patch adds a marker to the latest user content only when no message marker exists and a breakpoint slot remains. See Anthropic prompt caching.

Second, Anthropic hashes the prompt in tools, system, then messages order. Its documentation says, "A cache hit requires the prefix to match a recent request exactly, byte for byte." Hoisting a system message from the middle of messages[] into top-level system changes an earlier part of that hash and invalidates every cached message after it. Anthropic recommends appending the instruction as a role: "system" message instead because "The cached prefix stays the same." This patch preserves that structure for eligible Opus full-agent requests while retaining the legacy hoist elsewhere. See Anthropic mid-conversation system messages.

The cost difference is material. Anthropic prices a 5-minute cache write at 1.25 times the base input-token rate and a cache read at 0.1 times that rate. Rewriting a large prefix instead of reading it can cost 12.5 times as much for those tokens. See Anthropic prompt caching pricing.

Related issues

  • None.

Validation

  • Red/green request-path regression for an Opus 5 mid-conversation system cache breakpoint
  • Focused handler, cache-control, system-role, beta-flag, and Claude identity tests
  • npm run typecheck:core
  • ESLint passes for the changed production files
  • Prettier passes for all changed files
  • Production-code changes include a new or updated automated test in this PR
  • SonarQube PR analysis is green or any remaining issues are explicitly documented below
  • Production Docker image built successfully for the first commit
  • Isolated Opus 5 validation wrote 13,893 cache tokens, then read all 13,893 on the identical request

Tests added or updated

  • tests/unit/chatcore-translation-paths.test.ts
  • tests/unit/claude-code-parity.test.ts

Coverage notes

The request-path regression runs native Claude passthrough through the executor and inspects the final upstream body. It verifies that Opus 5 keeps the original message order, preserves the mid-conversation cache marker, does not hoist that block into top-level system, and does not add a redundant marker to the latest user turn.

The existing Sonnet and generic-client tests continue to cover the legacy hoisting path. The parity tests cover existing message markers and the four surviving-breakpoint limit.

Reviewer notes

The new behavior is limited to native Claude full-agent requests for supported Opus models. The same eligibility helper controls both the payload behavior and the Anthropic beta header, so unsupported request shapes keep the old normalization.

This PR does not change database state, migrations, or user settings.

@cryptiklemur
cryptiklemur marked this pull request as ready for review July 29, 2026 23:18
@cryptiklemur cryptiklemur changed the title fix(sse): supplement claude message cache breakpoint fix(sse): preserve Claude Code cache breakpoints Jul 30, 2026
@diegosouzapw
diegosouzapw merged commit de9fe1a into diegosouzapw:release/v3.8.50 Aug 6, 2026
3 checks passed
diegosouzapw added a commit that referenced this pull request Aug 6, 2026
…9554)

* fix(quality): reconcile inherited file-size drift on the release tip

13 files sit above their frozen LOC on the clean tip 8180b49
(measured by the gate itself). The PR-mode base-relative check (#8522)
correctly lets innocent PRs pass, but per-PR rebaselines were lost
across successive conflict resolutions of this hot file during the
08-05/06 merge batch — so the absolute mode (nightly, local runs) is
permanently red and stops distinguishing real growth from inherited
drift.

Frozen values updated to the measured tip, each annotated with the
merged PR that grew the file (#9024 #9324 #9329 #9193 #9332 #9228
#9236 #9314 #9260 #8934 #9196 #9163); executors default.ts and kiro.ts
(above the 1000 cap with no frozen entry) join the frozen set.

* fix(quality): prune orphaned ESLint suppressions and clear the 5 unsuppressed errors

The 'No new ESLint warnings' job reds the whole queue with exit 2:
'There are suppressions left that do not occur anymore' — the 08-05
merge batch removed code whose violations were frozen in
eslint-suppressions.json, leaving orphaned entries (673->670 files,
4338->4333 violations after eslint --prune-suppressions).

The full-tree run also surfaced 5 real unsuppressed errors merged with
the batch, fixed here instead of suppressed (new violations must be
fixed, per policy): 4x no-explicit-any in
tests/unit/catalog-order-contract.test.ts ((conn as any).id -> typed
cast) and 1x react/no-unescaped-entities in the agent-bridge
SetupWizard (#9095).

Also restores the _comment policy header the successive hot-file
conflict resolutions had dropped (TS7 debt freeze provenance + prune
policy).

* fix(quality): absorb the two file-size growths merged while this PR was in CI

The base kept moving during the reconcile cycle: #9184 grew
src/sse/handlers/chat.ts 1857->1877 and #9005 grew
open-sse/executors/default.ts 1027->1042. Re-measured on the merged
tree; gate back to 0 violations.

* fix(tests): move the orphaned RTL ratchet test to a collected path as node:test

#8828 added tests/unit/scripts/check-rtl-ratchet.test.ts — a path no
runner collects (the node:test globs enumerate an explicit subdir list
without scripts/, and vitest.config.ts never included it), so the file
NEVER ran and the test-discovery orphan gate reds the queue. Moved to
tests/unit/ (collected by node:test) and converted from vitest
describe/it/expect to node:test+assert to match the runner and the
sibling check-*.test.ts files. 5/5 green under the real runner.

---------

Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
Validated in local merge-train (devbox-vm-06-dev002) @ combined-tip (FAST gates green: static + changed tests + vitest — only pre-existing audit.test.ts flake). Evidence: /home/diegosouzapw/dev/proxys/OmniRoute/.claude/worktrees/merge-train-20260805-213228-suite.log
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…iegosouzapw#9554)

* fix(quality): reconcile inherited file-size drift on the release tip

13 files sit above their frozen LOC on the clean tip 30be158
(measured by the gate itself). The PR-mode base-relative check (diegosouzapw#8522)
correctly lets innocent PRs pass, but per-PR rebaselines were lost
across successive conflict resolutions of this hot file during the
08-05/06 merge batch — so the absolute mode (nightly, local runs) is
permanently red and stops distinguishing real growth from inherited
drift.

Frozen values updated to the measured tip, each annotated with the
merged PR that grew the file (diegosouzapw#9024 diegosouzapw#9324 diegosouzapw#9329 diegosouzapw#9193 diegosouzapw#9332 diegosouzapw#9228
diegosouzapw#9236 diegosouzapw#9314 diegosouzapw#9260 diegosouzapw#8934 diegosouzapw#9196 diegosouzapw#9163); executors default.ts and kiro.ts
(above the 1000 cap with no frozen entry) join the frozen set.

* fix(quality): prune orphaned ESLint suppressions and clear the 5 unsuppressed errors

The 'No new ESLint warnings' job reds the whole queue with exit 2:
'There are suppressions left that do not occur anymore' — the 08-05
merge batch removed code whose violations were frozen in
eslint-suppressions.json, leaving orphaned entries (673->670 files,
4338->4333 violations after eslint --prune-suppressions).

The full-tree run also surfaced 5 real unsuppressed errors merged with
the batch, fixed here instead of suppressed (new violations must be
fixed, per policy): 4x no-explicit-any in
tests/unit/catalog-order-contract.test.ts ((conn as any).id -> typed
cast) and 1x react/no-unescaped-entities in the agent-bridge
SetupWizard (diegosouzapw#9095).

Also restores the _comment policy header the successive hot-file
conflict resolutions had dropped (TS7 debt freeze provenance + prune
policy).

* fix(quality): absorb the two file-size growths merged while this PR was in CI

The base kept moving during the reconcile cycle: diegosouzapw#9184 grew
src/sse/handlers/chat.ts 1857->1877 and diegosouzapw#9005 grew
open-sse/executors/default.ts 1027->1042. Re-measured on the merged
tree; gate back to 0 violations.

* fix(tests): move the orphaned RTL ratchet test to a collected path as node:test

diegosouzapw#8828 added tests/unit/scripts/check-rtl-ratchet.test.ts — a path no
runner collects (the node:test globs enumerate an explicit subdir list
without scripts/, and vitest.config.ts never included it), so the file
NEVER ran and the test-discovery orphan gate reds the queue. Moved to
tests/unit/ (collected by node:test) and converted from vitest
describe/it/expect to node:test+assert to match the runner and the
sibling check-*.test.ts files. 5/5 green under the real runner.

---------

Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants