Skip to content

feat(status): hide cache section on backends that don't report it - #238

Merged
dylanneve1 merged 1 commit into
mainfrom
feat/cache-display-capability
May 21, 2026
Merged

feat(status): hide cache section on backends that don't report it#238
dylanneve1 merged 1 commit into
mainfrom
feat/cache-display-capability

Conversation

@claudiusthebot

Copy link
Copy Markdown
Collaborator

Summary

/status currently prints Cache 0% hit Β· Read 0 Write 0 on backends that don't report cache telemetry (Codex when no real cache hits, agy entirely), which reads as "the backend has no cache hits" when the truth is "the backend doesn't report cache usage at all." This adds a per-backend capability flag and renders nothing when there's nothing to render.

Changes

Capability flag on QueryBackend (src/core/types.ts):

cacheMetrics?: "none" | "read" | "readwrite";

Shared buildCacheDisplay() (src/frontend/status-context.ts): returns null when the backend's cacheMetrics === "none", otherwise a CacheDisplay { hitPct, read, write, showsWrite }. Frontends drop the entire Cache block when null.

Per-backend declarations (each factory.ts):

  • Claude SDK / Kilo / OpenCode β†’ "readwrite" (full telemetry)
  • Codex / OpenAI Agents / Antigravity β†’ "read" (provider reports cached input only, no write side)
  • agy β†’ "none" (Gemini CLI doesn't expose cache counters)

Frontends rewired β€” Telegram, Discord, Teams, Terminal all use buildCacheDisplay(). showsWrite controls whether the write count is rendered.

Hit-pct formula correction

While restructuring, caught a regression that the initial buildCacheDisplay would have shipped: the helper used read / (input + read + write) as the denominator. That's the pre-PR-#11 bug β€” cache_write is tokens being written to cache this turn, not served from it, and including them dilutes the hit ratio every time the cache is being warmed.

Now matches the canonical formula in src/backend/shared/usage.ts:cacheHitPercent and the existing Telegram inline calculation: read / (input + read). Three places, one rule.

Tests

src/__tests__/status-context.test.ts β€” 5 new cases:

  • cacheMetrics: "none" β†’ returns null (block hidden)
  • cacheMetrics: "read" β†’ write zeroed, showsWrite: false, correct hit-pct
  • cacheMetrics: "readwrite" β†’ explicit assertion that cacheWrite doesn't dilute hit-pct (this is the formula-regression guard)
  • Zero-effective-input edge case
  • Mode-"read" ignores any reported write count

status-context.test.ts 7/7 βœ… Β· terminal-renderer.test.ts 59/59 βœ… Β· npx tsc --noEmit clean Β· npm run lint no new warnings Β· prettier --check clean across touched files.

What this fixes in practice

On Codex with cached_input_tokens = 0 (typical mid-session before any prompt-cache hits land), /status previously displayed Cache 0% hit Β· Read 0 Write 0 β€” visually identical to "I have a working cache that just isn't being hit." Now Codex still shows the Cache block (it does report reads when they happen), but on agy β€” which exposes nothing β€” the block disappears entirely instead of lying.

For Claude SDK / Kilo / OpenCode the display is unchanged in behaviour (full read+write), but the hit-pct formula correction makes it match the shared usage helper and the historical 5560eea fix.

Closes the "Maybe we can open a PR so that the caching information will show dynamically if the backend advertises support for it" ask from chat earlier today.

Per-backend `cacheMetrics` capability flag added to the `QueryBackend`
interface: `"none" | "read" | "readwrite"`. Each backend declares what
it can actually surface, so /status stops printing fake "0% hit Β· Read 0
Write 0" on backends that don't report cache telemetry.

Shared `buildCacheDisplay()` in `src/frontend/status-context.ts`:
returns `null` when `cacheMetrics === "none"`, returns a `CacheDisplay`
otherwise. Frontends (Telegram, Discord, Teams, Terminal) drop the
Cache block entirely when null. `showsWrite` decides whether the write
count is rendered when the backend only reports reads.

While restructuring, fixed a regression in the cache-hit-% formula:
the new helper was using `read / (input + read + write)`, which dilutes
the hit ratio with cache_write tokens β€” those are tokens being *written
to* cache on this turn, not served from it. Now matches the canonical
formula in `src/backend/shared/usage.ts:cacheHitPercent`:
`read / (input + read)`. The shared usage helper, the old per-frontend
inline calc on Telegram, and now `buildCacheDisplay` all agree.

Backend declarations:
- Claude SDK / Kilo / OpenCode β†’ `"readwrite"` (full telemetry)
- Codex / OpenAI Agents / Antigravity β†’ `"read"` (provider reports
  cached input only, no write side)
- agy β†’ `"none"` (Gemini CLI doesn't expose cache counters)

Tests added in `status-context.test.ts` cover all three modes plus
the zero-input edge case, and explicitly assert the canonical hit-pct
formula (was previously encoding the buggy 29% for readwrite mode).
@dylanneve1
dylanneve1 enabled auto-merge (squash) May 21, 2026 17:56
@dylanneve1
dylanneve1 merged commit 7c606bf into main May 21, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants