fix(usage): include cache tokens in input token counts by tombii · Pull Request #822 · cheaper-inference/OmniRoute

tombii · 2026-03-30T16:23:34Z

Summary

Fix getLoggedInputTokens to return full prompt_tokens (input + cache_read + cache_creation) per Claude API docs
Fix usageExtractor for non-streaming Claude responses to calculate total correctly
Add formatUsageLog helper to show CR=<cache_read> in logs
Add migration 012 to fix historical token counts in usage_history
Move prompt cache metrics from Settings → AI tab to /dashboard/cache page

Problem

Cache metrics showed totalInputTokens (71,907) was less than totalCacheCreationTokens (1,350,674), which is impossible. This was because:

getLoggedInputTokens() was incorrectly subtracting cached tokens from prompt_tokens
Non-streaming responses weren't calculating the full token total
Historical data was stored incorrectly in the database

Solution

Per Claude API documentation:

Total input tokens = input_tokens + cache_creation_input_tokens + cache_read_input_tokens

Code Fixes

src/lib/usage/tokenAccounting.ts: Return full prompt_tokens without subtraction
open-sse/handlers/usageExtractor.ts: Calculate promptTokens = inputTokens + cacheRead + cacheCreation
open-sse/handlers/chatCore.ts: Show cache read tokens in logs as CR=<value>
Added migration 012_fix_token_input_cache_tokens.sql (idempotent)
Fixed t.rich() hydration error in cache page

UX Improvement

Moved CacheStatsCard from /dashboard/settings?tab=ai to /dashboard/cache so all cache metrics (semantic + prompt cache) are in one unified location.

Test Results

✅ All 1157 unit tests pass
✅ Cache metrics API returns correct totals
✅ Migration is idempotent and tracked in _omniroute_migrations
✅ Logs show cache read tokens: in=6055 | out=211 | CR=22399
✅ Fixed hydration mismatch error in cache page

Before/After

Before:

{
  "totalInputTokens": 71190,
  "totalCachedTokens": 2298850,
  "totalCacheCreationTokens": 1350674
}

After:

{
  "totalInputTokens": 3720714,
  "totalCachedTokens": 2298850,
  "totalCacheCreationTokens": 1350674
}

Cache read ratio now correctly shows 61% instead of an impossible 3229%.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com

kilo-code-bot · 2026-03-30T16:25:40Z

+
+-- Update tokens_input to include cache tokens (only if not already done)
+UPDATE usage_history
+SET tokens_input = tokens_input + tokens_cache_read + tokens_cache_creation


WARNING: Migration logic may be flawed. The condition tokens_input < tokens_cache_read + tokens_cache_creation can produce incorrect results for edge cases:

If base input_tokens was very small (e.g., 10) and cache_read is large (e.g., 4000), the condition 10 < 4000 = true would allow migration, but the sum would incorrectly double-add

If tokens_input already includes cache tokens correctly, but happens to be numerically smaller than the sum, this would add them again

Recommendation: Add a version flag or timestamp column to track whether migration has been applied, rather than trying to infer from the data values.

kilo-code-bot · 2026-03-30T16:25:50Z

Code Review Summary

Status: Issues Resolved | Recommendation: Merge

Overview

Severity	Count
CRITICAL	0
WARNING	0
SUGGESTION	0

Previous Issues - RESOLVED

All issues from the previous review have been addressed:

Migration SQL (lines 13-15) - ✅ RESOLVED: The problematic idempotency condition tokens_input < tokens_cache_read + tokens_cache_creation has been removed. The migration now simply updates all records where cache tokens exist, which is correct since migrations are tracked in _omniroute_migrations table.
Unused import (settings/page.tsx) - ✅ RESOLVED: The CacheStatsCard import and its usage have both been removed from the file. The component is now correctly located only in dashboard/cache/page.tsx.

Files Reviewed (7 files)

open-sse/handlers/chatCore.ts - Added formatUsageLog import and usage
open-sse/handlers/usageExtractor.ts - Fixed Claude token calculation to include cache tokens in total
src/app/(dashboard)/dashboard/cache/page.tsx - Added CacheStatsCard component; fixed t.rich() hydration error
src/app/(dashboard)/dashboard/settings/page.tsx - Removed CacheStatsCard (moved to cache page)
src/lib/db/migrations/012_fix_token_input_cache_tokens.sql - Migration to fix historical token data
src/lib/usage/tokenAccounting.ts - Core token accounting changes; added formatUsageLog helper
tests/integration/integration-wiring.test.mjs - Updated tests for new page structure

Assessment

The PR correctly implements the fix for Claude token accounting per API documentation. All code changes are appropriate, and the migration logic is now correct. No new issues found.

gemini-code-assist

Code Review

This pull request updates token accounting to correctly include Claude's cache tokens in the total prompt token count, refactors usage logging, and moves the CacheStatsCard to the cache dashboard. A critical issue was found in the database migration where the idempotency check in the WHERE clause is logically flawed and could cause the migration to skip records that require updates.

gemini-code-assist · 2026-03-30T16:25:56Z

+UPDATE usage_history
+SET tokens_input = tokens_input + tokens_cache_read + tokens_cache_creation
+WHERE (tokens_cache_read > 0 OR tokens_cache_creation > 0)
+  AND tokens_input < tokens_cache_read + tokens_cache_creation;


The idempotency check in the WHERE clause is flawed and will cause the migration to silently skip some records that need to be updated.

The condition AND tokens_input < tokens_cache_read + tokens_cache_creation assumes that for any un-migrated record, the number of base input tokens is less than the sum of cached tokens. This assumption is not always valid. For example, a request could have a large number of new input tokens and only a small number of cached tokens, causing this condition to be false and preventing the record from being corrected.

Since the pull request description mentions that migrations are tracked in _omniroute_migrations, the migration runner likely ensures that each migration is executed only once. If that's the case, this manual idempotency check is not needed and should be removed to ensure all relevant historical records are fixed.

UPDATE usage_history SET tokens_input = tokens_input + tokens_cache_read + tokens_cache_creation WHERE (tokens_cache_read > 0 OR tokens_cache_creation > 0);

kilo-code-bot · 2026-03-30T16:46:54Z

@@ -88,7 +88,6 @@ export default function SettingsPage() {
              <ThinkingBudgetTab />


WARNING: Unused import - The CacheStatsCard import at line 20 remains but the component is no longer used here (moved to dashboard/cache/page.tsx). The import should be removed.

- Fix getLoggedInputTokens to return full prompt_tokens (input + cache_read + cache_creation) - Fix usageExtractor for non-streaming Claude responses to calculate total correctly - Add formatUsageLog helper to show CR=<cache_read> in logs - Add migration 012 to fix historical token counts in usage_history - Move prompt cache metrics from Settings to /dashboard/cache page Per Claude API docs: Total input tokens = input_tokens + cache_creation_input_tokens + cache_read_input_tokens Fixes issue where totalInputTokens (71k) was less than totalCacheCreationTokens (1.35M). Tested: - All 1134 unit tests pass - Cache metrics API returns correct totals - Migration is idempotent and tracked in _omniroute_migrations - Logs show cache read tokens: 'in=6055 | out=211 | CR=22399' Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Integration test was failing because CacheStatsCard was moved from settings page to cache page in previous commit. Split the test into two separate describe blocks for accurate page-specific verification. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

diegosouzapw · 2026-03-30T21:17:48Z

Thanks @tombii for this critical fix! 🎉 The token accounting improvement allows precise cost tracking. This has been approved and will be shipped in the incoming release. Highly appreciated!

tombii · 2026-03-30T21:20:09Z

I'm just solving the merge conflict, or did you merge it already?

tombii · 2026-03-30T21:39:21Z

Follow-up PR #835 adds UI polish for the cache page:\n- Consistent header sizing\n- Request count context in cache hit rate labels

kilo-code-bot Bot reviewed Mar 30, 2026

View reviewed changes

gemini-code-assist Bot reviewed Mar 30, 2026

View reviewed changes

tombii force-pushed the fix/cache-token-accounting branch 2 times, most recently from ce14278 to e9656c6 Compare March 30, 2026 16:37

kilo-code-bot Bot reviewed Mar 30, 2026

View reviewed changes

tombii force-pushed the fix/cache-token-accounting branch from e9656c6 to c6eadc5 Compare March 30, 2026 17:24

diegosouzapw merged commit 76e1350 into cheaper-inference:main Mar 30, 2026
1 check passed

tombii mentioned this pull request Mar 30, 2026

fix(ui): improve cache page header sizing and context #835

Merged

diegosouzapw mentioned this pull request Apr 22, 2026

Release v3.7.0 #1439

Merged

htkarabulut mentioned this pull request May 29, 2026

Fix analytics history and log token accounting #2904

Merged

2 of 5 tasks

diegosouzapw mentioned this pull request Jul 20, 2026

fix(mitm): gate Agent Bridge Repair on sudo password (#7836) #7865

Merged

3 of 5 tasks

Navigation Menu

Repository navigation

fix(usage): include cache tokens in input token counts - #822

fix(usage): include cache tokens in input token counts#822
diegosouzapw merged 2 commits into
cheaper-inference:mainfrom
tombii:fix/cache-token-accounting

tombii commented Mar 30, 2026

Uh oh!

kilo-code-bot Bot Mar 30, 2026

Uh oh!

kilo-code-bot Bot commented Mar 30, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Mar 30, 2026

Uh oh!

kilo-code-bot Bot Mar 30, 2026

Uh oh!

diegosouzapw commented Mar 30, 2026

Uh oh!

tombii commented Mar 30, 2026

Uh oh!

Uh oh!

tombii commented Mar 30, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		@@ -88,7 +88,6 @@ export default function SettingsPage() {
		<ThinkingBudgetTab />

Conversation

tombii commented Mar 30, 2026

Summary

Problem

Solution

Code Fixes

UX Improvement

Test Results

Before/After

Uh oh!

kilo-code-bot Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

kilo-code-bot Bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Summary

Overview

Previous Issues - RESOLVED

Files Reviewed (7 files)

Assessment

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

kilo-code-bot Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

diegosouzapw commented Mar 30, 2026

Uh oh!

tombii commented Mar 30, 2026

Uh oh!

Uh oh!

tombii commented Mar 30, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kilo-code-bot Bot commented Mar 30, 2026 •

edited

Loading