Repository navigation
fix(sse): advance Claude cache breakpoints on growing tails - #10684
diegosouzapw merged 77 commits into
Conversation
f0438f9 to
c8efc80
Compare
|
Thanks for this fix — advancing the prompt-cache breakpoint onto the growing conversation tail closes a real gap (the current ensureCacheControlOnLastUserMessage keeps an existing marker pinned at the first turn, so the cached prefix stops growing on long sessions). A couple of things to confirm before merge: (1) rebase against the current release/v3.8.50 tip — the PR is currently marked as conflicting; (2) please add/keep a unit test covering a multi-turn payload asserting the marker is advanced to the final user message without exceeding the 4-breakpoint cap, and that the existing #9436 boundary tests (both hoist paths + native Claude path) still pass; (3) ensure no double-advance leaves duplicate markers and that TTL ordering (1h-before-5m) is preserved. Nice, tightly-scoped fix. |
8c2d900 to
71cf84c
Compare
71cf84c to
8262395
Compare
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…10627) (#10709) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…new cert (#10467) (#10715) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…ted seeds, identity faces (#10716) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
… Windows (#10717) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…0722) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…10725) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…through (#10729) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…cks (#10681) (#10730) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…ken refresh (#10732) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…tats (#10769) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…log (#10770) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…10772) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
…cription provider (#10773) (#10774) Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Merged — validated together with a batch of related RaviTharuma PRs in one combined worktree (typecheck:core clean, complexity/file-size/changelog gates green, focused tests passing). Thanks for the contribution!
Merged — validated together with a batch of related RaviTharuma PRs in one combined worktree (typecheck:core clean, complexity/file-size/changelog gates green, focused tests passing). Thanks for the contribution!
…S heap (#10818) Merged — validated together with a batch of related RaviTharuma PRs in one combined worktree (typecheck:core clean, complexity/file-size/changelog gates green, focused tests passing). Thanks for the contribution!
Merged — validated together with a batch of related RaviTharuma PRs in one combined worktree (typecheck:core clean, complexity/file-size/changelog gates green, focused tests passing). Thanks for the contribution!
…10814) Merged — validated together with a batch of related RaviTharuma PRs in one combined worktree (typecheck:core clean, complexity/file-size/changelog gates green, focused tests passing). Thanks for the contribution!
Merged — validated together with a batch of related RaviTharuma PRs in one combined worktree (typecheck:core clean, complexity/file-size/changelog gates green, focused tests passing). Thanks for the contribution!
Merged — locally validated (changelog gate green) after resolving base-drift against #10817's SQLite HA section (both landed today, same insertion point in DOCKER_GUIDE.md — combined, both sections kept). Thanks!
Merged — carried forward the PR's own real value (the first 2 commits: ignore ad-hoc BOT_TOKEN/BOT_URL in env-doc-sync, plus the lock-in test). The branch had accumulated 7 more commits chasing the moving release tip across several rebases (each one re-fixing base-reds that had already moved again by the next rebase) — dropped those since they no longer apply to the current tip, and cherry-picked just the 2 with lasting value, preserving your authorship. 14/14 focused tests pass, changelog gate green. Thanks!
scripts/ad-hoc mesh helpers read operator-supplied BOT_TOKEN/BOT_URL. They are not OmniRoute runtime config and should not fail Docs Gates on every PR. Unblocks check:env-doc-sync on release/v3.8.50.
Drive runEnvDocSync so BOT_TOKEN/BOT_URL are ignored by IGNORE_FROM_CODE and flagged when ignore is empty.
#10838) Merged — locally validated (60/60 combined image-generation tests, typecheck:core clean, complexity/cognitive/file-size/changelog gates green).
…10842) Merged — locally validated (60/60 combined image-generation tests, typecheck:core clean, complexity/cognitive/file-size/changelog gates green, small rebaseline for the combined imageRegistry.ts growth).
Merged — locally validated together with related cryptiklemur PRs (typecheck:core clean, complexity/cognitive/file-size/changelog gates green, focused tests passing). Real bug, clean fix, great regression test. Thanks!
Merged — locally validated together with related cryptiklemur PRs (typecheck:core clean, gates green). Good catch on the phantom usage_logs table. Thanks!
Merged — locally validated (typecheck:core clean, gates green). Thanks for the extra regression coverage!
Merged — locally validated (70/70 combined chatcore-translation-paths tests, typecheck:core clean, gates green). Thanks!
22e46a0
into
diegosouzapw:release/v3.8.50
…s before cached fallback Two release/v3.8.50 base-red unit failures in tests/unit/cc-compatible-provider.test.ts (part of the diegosouzapw#9985 "release branch not green" set), re-derived and fixed against the current tip rather than cherry-picked, since the base-red set has drifted since the originating PRs were written. 1. models route (genuine code bug; ref diegosouzapw#10795 fix diegosouzapw#1, closes the gap in diegosouzapw#10863): GET /api/providers/:id/models returned HTTP 200 with an empty/cached model list for a Claude-Code-compatible (anthropic-compatible-cc-*) connection instead of the documented HTTP 400 "does not support models listing". The isClaudeCodeCompatibleProvider check sat AFTER maybeReturnCachedDiscovery() / maybeReturnAutoFetchDisabled() inside the isAnthropicCompatibleProvider block, so a freshly-created CC connection with no synced models hit the auto-fetch-disabled 200 fallback first. Hoisted the CC 400 check to the top of that block so the unsupported-listing contract always wins. 2. stale test assertion (test-only; the code is correct per diegosouzapw#10684): the "preserves client cache markers" test asserted a client message marker sent with no ttl defaults to 1h even when a system block carries an explicit 5m marker. diegosouzapw#10684 ("advance Claude cache breakpoints on growing tails") changed normalizeCacheControlTtl to "default missing ttl to 5m after a 5m breakpoint" (sections processed tools -> system -> messages) and updated claude-code-parity.test.ts / chatcore-translation-paths.test.ts to match, but missed this third assertion. Updated it to expect 5m, matching the intended breakpoint semantics. Verified: tests/unit/cc-compatible-provider.test.ts 27/27, claude-code-parity 40/40, chatcore-translation-paths 75/75, typecheck:core clean. Refs diegosouzapw#9985, diegosouzapw#10795, diegosouzapw#10863, diegosouzapw#10684.
The 8c4a219 revert of diegosouzapw#10828 accidentally re-scrambled the ordering inside the isAnthropicCompatibleProvider branch of the provider models route: the cached-discovery / auto-fetch-disabled fallbacks ran BEFORE the isClaudeCodeCompatibleProvider rejection, so a CC node with no cached catalog got a misleading 200 {models: [], warning: "Auto-fetch disabled"} instead of the contractual 400 "does not support models listing". Restore diegosouzapw#10828's original order — CC rejection first — and update the cache-marker ttl expectation for diegosouzapw#10684's 5m-inheritance rule in the same file's test. 27/27 green.
The 8c4a219 revert of diegosouzapw#10828 accidentally re-scrambled the ordering inside the isAnthropicCompatibleProvider branch of the provider models route: the cached-discovery / auto-fetch-disabled fallbacks ran BEFORE the isClaudeCodeCompatibleProvider rejection, so a CC node with no cached catalog got a misleading 200 {models: [], warning: "Auto-fetch disabled"} instead of the contractual 400 "does not support models listing". Restore diegosouzapw#10828's original order — CC rejection first — and update the cache-marker ttl expectation for diegosouzapw#10684's 5m-inheritance rule in the same file's test. 27/27 green.
…s before cached fallback Two release/v3.8.50 base-red unit failures in tests/unit/cc-compatible-provider.test.ts (part of the diegosouzapw#9985 "release branch not green" set), re-derived and fixed against the current tip rather than cherry-picked, since the base-red set has drifted since the originating PRs were written. 1. models route (genuine code bug; ref diegosouzapw#10795 fix diegosouzapw#1, closes the gap in diegosouzapw#10863): GET /api/providers/:id/models returned HTTP 200 with an empty/cached model list for a Claude-Code-compatible (anthropic-compatible-cc-*) connection instead of the documented HTTP 400 "does not support models listing". The isClaudeCodeCompatibleProvider check sat AFTER maybeReturnCachedDiscovery() / maybeReturnAutoFetchDisabled() inside the isAnthropicCompatibleProvider block, so a freshly-created CC connection with no synced models hit the auto-fetch-disabled 200 fallback first. Hoisted the CC 400 check to the top of that block so the unsupported-listing contract always wins. 2. stale test assertion (test-only; the code is correct per diegosouzapw#10684): the "preserves client cache markers" test asserted a client message marker sent with no ttl defaults to 1h even when a system block carries an explicit 5m marker. diegosouzapw#10684 ("advance Claude cache breakpoints on growing tails") changed normalizeCacheControlTtl to "default missing ttl to 5m after a 5m breakpoint" (sections processed tools -> system -> messages) and updated claude-code-parity.test.ts / chatcore-translation-paths.test.ts to match, but missed this third assertion. Updated it to expect 5m, matching the intended breakpoint semantics. Verified: tests/unit/cc-compatible-provider.test.ts 27/27, claude-code-parity 40/40, chatcore-translation-paths 75/75, typecheck:core clean. Refs diegosouzapw#9985, diegosouzapw#10795, diegosouzapw#10863, diegosouzapw#10684.
…s before cached fallback Two release/v3.8.50 base-red unit failures in tests/unit/cc-compatible-provider.test.ts (part of the diegosouzapw#9985 "release branch not green" set), re-derived and fixed against the current tip rather than cherry-picked, since the base-red set has drifted since the originating PRs were written. 1. models route (genuine code bug; ref diegosouzapw#10795 fix diegosouzapw#1, closes the gap in diegosouzapw#10863): GET /api/providers/:id/models returned HTTP 200 with an empty/cached model list for a Claude-Code-compatible (anthropic-compatible-cc-*) connection instead of the documented HTTP 400 "does not support models listing". The isClaudeCodeCompatibleProvider check sat AFTER maybeReturnCachedDiscovery() / maybeReturnAutoFetchDisabled() inside the isAnthropicCompatibleProvider block, so a freshly-created CC connection with no synced models hit the auto-fetch-disabled 200 fallback first. Hoisted the CC 400 check to the top of that block so the unsupported-listing contract always wins. 2. stale test assertion (test-only; the code is correct per diegosouzapw#10684): the "preserves client cache markers" test asserted a client message marker sent with no ttl defaults to 1h even when a system block carries an explicit 5m marker. diegosouzapw#10684 ("advance Claude cache breakpoints on growing tails") changed normalizeCacheControlTtl to "default missing ttl to 5m after a 5m breakpoint" (sections processed tools -> system -> messages) and updated claude-code-parity.test.ts / chatcore-translation-paths.test.ts to match, but missed this third assertion. Updated it to expect 5m, matching the intended breakpoint semantics. Verified: tests/unit/cc-compatible-provider.test.ts 27/27, claude-code-parity 40/40, chatcore-translation-paths 75/75, typecheck:core clean. Refs diegosouzapw#9985, diegosouzapw#10795, diegosouzapw#10863, diegosouzapw#10684. (cherry picked from commit 6746069)
…s before cached fallback Two release/v3.8.50 base-red unit failures in tests/unit/cc-compatible-provider.test.ts (part of the diegosouzapw#9985 "release branch not green" set), re-derived and fixed against the current tip rather than cherry-picked, since the base-red set has drifted since the originating PRs were written. 1. models route (genuine code bug; ref diegosouzapw#10795 fix diegosouzapw#1, closes the gap in diegosouzapw#10863): GET /api/providers/:id/models returned HTTP 200 with an empty/cached model list for a Claude-Code-compatible (anthropic-compatible-cc-*) connection instead of the documented HTTP 400 "does not support models listing". The isClaudeCodeCompatibleProvider check sat AFTER maybeReturnCachedDiscovery() / maybeReturnAutoFetchDisabled() inside the isAnthropicCompatibleProvider block, so a freshly-created CC connection with no synced models hit the auto-fetch-disabled 200 fallback first. Hoisted the CC 400 check to the top of that block so the unsupported-listing contract always wins. 2. stale test assertion (test-only; the code is correct per diegosouzapw#10684): the "preserves client cache markers" test asserted a client message marker sent with no ttl defaults to 1h even when a system block carries an explicit 5m marker. diegosouzapw#10684 ("advance Claude cache breakpoints on growing tails") changed normalizeCacheControlTtl to "default missing ttl to 5m after a 5m breakpoint" (sections processed tools -> system -> messages) and updated claude-code-parity.test.ts / chatcore-translation-paths.test.ts to match, but missed this third assertion. Updated it to expect 5m, matching the intended breakpoint semantics. Verified: tests/unit/cc-compatible-provider.test.ts 27/27, claude-code-parity 40/40, chatcore-translation-paths 75/75, typecheck:core clean. Refs diegosouzapw#9985, diegosouzapw#10795, diegosouzapw#10863, diegosouzapw#10684. (cherry picked from commit 6746069)
…zapw#10684) Merged — locally validated (110/110 focused tests, typecheck:core clean, gates green) after resolving base-drift against diegosouzapw#10683 (both landed today, same test file — combined). Nice, well-tested cache-breakpoint fix. Thanks!
CI note:
Docs Gatescurrently inherits the release-baseBOT_TOKEN/BOT_URLenv-doc failure. the same base-red appears on unrelated #10780; this diff does not touch those variables.Claude Code requests stopped advancing the cached prefix after an older message breakpoint appeared. later turns stayed outside the reusable prefix even as the conversation grew.
Anthropic requires longer TTL breakpoints before shorter ones and checks the prompt in tools, system, then messages order: Prompt caching.
what changed
5mafter a5mboundary, preserving1hbefore5mThe multi-turn tests cover the final user message, the four-breakpoint cap, repeated calls, both hoist paths, and native Claude passthrough.
how to test
npx cross-env DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=4096 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=1 tests/unit/claude-code-parity.test.ts npx cross-env DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=4096 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-isolation=none --test-concurrency=1 --test-name-pattern='chatCore (builds Claude Code-compatible|normalizes native Claude Code messages|preserves Opus 5 mid-conversation|keeps Claude normalization|preserves cache_control automatically|advances a message cache breakpoint)' tests/unit/chatcore-translation-paths.test.tsBoth focused runs pass after rebasing onto the current
release/v3.8.50tip.