[amirimatin]

Summary

Claude cache writes can be explicitly excluded from OmniRoute's internal Chat Completions prompt total (cache_creation_in_prompt: false). The Responses conversion preserves those details but previously emitted the exclusive total as input_tokens. Desktop clients therefore received a severely understated context size on a cold cache: a fixture with 769 prompt tokens and 170,215 cache-creation tokens reported 769 input tokens instead of 170,984.

Normalize explicitly excluded cache writes at the Responses output boundary, adding them to input_tokens and total_tokens and marking the emitted details inclusive. Cover JSON responses, native Responses SSE snapshots, the live translator, and the legacy SSE transformer. Warm-cache, already-inclusive, and unmarked usage retain their existing totals; repeat normalization is idempotent. Keep internal stream accounting and multichunk aggregation in upstream units.

Target Branch

• Targets the active release/v3.8.53 branch, reconciled against d1ba82234c.
• Originally reproduced and patched against release/v3.8.52; the upstream source fix has been rebased onto the current development release.

Proof Of Fix

• TDD regression tests fail before the fix and pass afterward.
• Initial cold-cache regression: actual: 769, expected: 170984; mixed read/write regression: 110 !== 130.
• Additional regressions cover repeated sanitization, later usage chunks without repeated cache details, output-only updates, and unchanged internal accounting.
• An isolated runtime replay against the patched 3.8.52 image also returned HTTP 200 and reported 171,078 input tokens, matching 842 uncached + 170,236 cache-creation tokens. Deployment-specific image artifacts are outside this PR.

Tests Added Or Updated

• tests/unit/responses-context-cache-creation.test.ts: new JSON/native/live/legacy SSE regression matrix and multichunk cases.
• tests/unit/cache-creation-marker.test.ts: update the Responses output contract to include context while retaining the assertions for internal and Chat Completions accounting.

Validation

• Focused suites: 103 passed, zero failures after reconciliation:

node --import tsx/esm --test \
  tests/unit/responses-context-cache-creation.test.ts \
  tests/unit/response-sanitizer.test.ts \
  tests/unit/issue-13472-responses-cache-creation.test.ts \
  tests/unit/translator-responses-cache-usage.test.ts \
  tests/unit/responses-usage-trailing-6906.test.ts \
  tests/unit/cache-creation-marker.test.ts

• ESLint on all six changed files; Prettier; git diff --check.
• TypeScript check of the new standalone helper.
• Pre-commit gates, including manifest, documentation-version sync, any-budget, and tracked-artifact checks. The private UI-kit snapshot gate skips when its private checkout is absent.
• Full npm run lint: exits 2 because the repository suppression file contains entries that no longer occur. The command reported no individual lint violations; all six changed files pass focused ESLint. No global suppressions were modified in this PR.
• Full unit/Vitest matrix, coverage ratchet, core typecheck, and production source build: CI validation pending. The fork-triggered CI/Quality Gates runs currently report action_required, with no jobs started.

Reviewer Notes

Only an explicit cache_creation_in_prompt: false with finite, positive cache-creation tokens triggers adjustment. The inclusion marker is flipped on a copied result, so raw usage state and upstream billing semantics are preserved. No request, model, cooldown, or error-status behavior changes.

This fixes a reproduced context-accounting defect. It does not establish or claim to fix the cause of empty/thinking-only upstream responses or subsequent model-cooldown 429s.

[coderabbitai]

No actionable comments were generated in the recent review.

Summary

Summary by CodeRabbit

• Bug Fixes
  • Responses API usage now includes cache-creation tokens in input and total counts when those tokens were not already included.
  • Streaming and non-streaming responses apply this accounting consistently, while avoiding double-counting at repeated processing steps.
  • Cache details remain available, and logged input-token counts retain their original units.

Responses usage now includes eligible cache-creation tokens in its input and total counts. Sanitization and completed-response paths apply the shared normalization helper. Tests cover native and translated Responses usage, repeated sanitization, and later legacy SSE usage updates.

Changes: Responses usage accounting

Layer / File(s)	Summary
Cache-creation usage normalization open-sse/utils/normalizeResponsesContextUsage.ts, tests/unit/responses-context-cache-creation.test.ts	The helper adds cache-creation tokens to input and total counts when the details mark them excluded and the token counts are valid. It marks cache creation as included. Tests cover usage fixtures and shared expected-count assertions.
Sanitized Responses usage open-sse/handlers/responseSanitizer.ts, tests/unit/responses-context-cache-creation.test.ts	Non-streaming and streaming sanitization normalize eligible usage objects. Tests cover sanitization, repeated sanitization, and usage handling across response paths.
Completed response usage open-sse/translator/response/openai-responses.ts, open-sse/transformer/responsesTransformer.ts, tests/unit/cache-creation-marker.test.ts, tests/unit/responses-context-cache-creation.test.ts	The translator and transformer normalize usage when building completed responses. Tests check inclusive input and total counts, marker values, and later legacy SSE usage updates.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Merge Risk: Minimal · up to 4b679

No merge-blocking risk was established for the usage normalization change.

Security architecture risk: Low · up to 4b679

The change has limited security impact: it adjusts reported context size without granting additional access. The inspected streaming paths preserve internal accounting, but coverage of native passthrough and exceptional accounting fallbacks remains incomplete.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — The demonstrated changed exposure is Responses usage metadata for requests whose provider-derived usage explicitly excludes cache writes. The inspected change does not establish broader tenant, credential or service authority.

Trust Boundaries and Controls

• observed — The inspected tool-interception function selects executable calls using tool identity and server execution context. The new normalizer changes usage totals and their inclusion marker, not those tool identities or execution-context fields.

Resilience and Maintainability Implications

• observed — Copy-on-normalization and the inclusive marker prevent emitted totals from contaminating upstream-unit stream aggregation or accumulating cache writes on repeated normalization.

Pre-merge checks: 6 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 6 passed

Check name	Status	Explanation
Title check		The title follows the Conventional Commits format with type "fix" and scope "responses". It names the main change: explicitly excluded cache writes are now included in Responses context usage. This ma…
Description check		The description covers the required template sections: Summary, Related Issues (not filled in), Target Branch, Validation, Tests Added Or Updated, and Reviewer Notes. Coverage Notes is absent. The Rel…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Tests For Production Code		Production code changed under open-sse/. The PR also adds tests/unit/responses-context-cache-creation.test.ts (+252 lines) and updates tests/unit/cache-creation-marker.test.ts (+18/-9). The tests exer…
No Ai Attribution		PASS. Both commits in the reviewed range have no commit trailers or AI-generation footers. The supplied PR body contains no Co-Authored-By trailer naming an AI or bot and no `Generated with <AI tool…

Full details: Title check

Explanation

The title follows the Conventional Commits format with type "fix" and scope "responses". It names the main change: explicitly excluded cache writes are now included in Responses context usage. This matches the raw summary, the description, and the objectives.

Full details: Description check

Explanation

The description covers the required template sections: Summary, Related Issues (not filled in), Target Branch, Validation, Tests Added Or Updated, and Reviewer Notes. Coverage Notes is absent. The Related Issues section has no issue number, and the template checkboxes are only partly ticked. The description is mostly complete and clearly explains the change, the tests, and the validation status, so the check passes.

Full details: Tests For Production Code

Explanation

Production code changed under open-sse/. The PR also adds tests/unit/responses-context-cache-creation.test.ts (+252 lines) and updates tests/unit/cache-creation-marker.test.ts (+18/-9). The tests exercise native SSE, non-streaming sanitization, live SSE translation, legacy transformation, idempotence, and cache-token accounting for the changed behavior.

Full details: No Ai Attribution

Explanation

PASS. Both commits in the reviewed range have no commit trailers or AI-generation footers. The supplied PR body contains no Co-Authored-By trailer naming an AI or bot and no Generated with <AI tool> footer. The human commit authors are permitted.

Full details: Docstring Coverage

Explanation

Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 6 functions across 6 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches

Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Recent review info

Run configuration

• Source mode: Auto
• Repository: cheaper-inference/OmniRoute
• Branch: release/v3.8.53
• File: .coderabbit.yaml
• Commit: f000c11
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Advanced
• Run ID: 146e58f4-fdb6-41bb-bbb7-76ece2b0a0db

Commits

Reviewing files that changed from the base of the PR and between f000c11 and 4b679e3.

Files selected for processing: 6

• open-sse/handlers/responseSanitizer.ts
• open-sse/transformer/responsesTransformer.ts
• open-sse/translator/response/openai-responses.ts
• open-sse/utils/normalizeResponsesContextUsage.ts
• tests/unit/cache-creation-marker.test.ts
• tests/unit/responses-context-cache-creation.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.