fix(desktop): move the context meter onto the provider's real token count - #332
Merged
Merged
Conversation
…ount The status-bar meter still lags through a long agentic turn. Observed 2026-08-02: it sat at 55.8k while the server was serving that same session at 65.7k+ across in-turn tool rounds. #316 made a completed TOOL push a `token.usage` frame, which is what got the gauge moving mid-turn at all. But a tool completion is the wrong signal for the quantity being published. It reports the request that *produced* that tool call, before its result was appended, so the reading is a full round trip behind whatever is in flight — and a round that ends without calling a tool (a reasoning-only round, a `length` continuation, the final round of every turn) says nothing at all. When several of those land in a row the gauge does not drift, it sits still, which is what "pinned" looked like. That last step is an argument from the code, not a reproduction: the harness cannot currently drive a turn that keeps going without calling a tool, so the field symptom itself was not reproduced here. What is reproduced, and asserted, is that occupancy now reaches the wire on the response rather than on the tool — see the verification note at the end for exactly where the line is drawn. Usage now goes out from `record_canonical_usage`, the one place that sees the provider's own prompt count land on the compressor. That is one frame per usage-bearing API response, on the real number rather than an estimate, and because both the chat loop and the codex app-server runtime account through it, both are covered by the same three lines. #316's tool-completion sample stays as a cheap second one — with de-duplication it costs nothing, and it still covers a runtime that accounts outside that recorder. The agent loop had offered a hook for exactly this since the live context bar was added, and nothing ever assigned it, so the branch was dead. It was also offered at the wrong moment — *before* the request, carrying `estimate_messages_tokens_rough`. That estimate assumes ~4 chars/token while dense tool-call/JSON transcripts tokenize nearer ~3.4, so it reads materially low against the server: the over-reservation note in `model_metadata` measured est 58,039 against server >=65,797 on deepseek-v4-flash-w2, a gap the same shape and size as the one reported here. Publishing it would have put a number on the meter that under-reads the window by ~13%. The estimate still does its real job (sizing compression decisions); it just no longer pretends to be a measurement, and the helper is renamed to `_raise_preflight_context_estimate` to stop the name promising otherwise. Three details that matter more than the wiring: - **The hook is a post-construction attribute, never an AIAgent kwarg.** `delegate_tool` forwards ~30 constructor kwargs from parent to child, so a kwarg is precisely how a subagent would inherit this and start reporting its own much smaller window on the parent's gauge. Auxiliary calls — goal judge, title, compaction — never reach the recorder at all; they account through `record_aux_usage`, which touches neither the compressor nor the session counters. Both isolations are asserted, the goal judge through the real auxiliary path with its recorded row checked, so the test cannot pass by the call simply never happening. - **Frames are de-duplicated and floored at 1/s.** Every tool in a parallel batch reads the same occupancy, so an unchanged gauge is dropped outright and costs nothing. A moved gauge waits out the floor; dropping one is safe because occupancy only climbs within a turn, so a later frame supersedes it and `message.complete` still carries the turn's final number. A compaction is exempt — it is the one legitimate fall, and it is the drop the client's monotonic guard is watching `compressions` to authorize. - **The usage frame now precedes the tool card.** `tool.complete` is the one emit in that callback not wrapped in `try/except`; a tool result that fails to serialize raises through it and the tool executor swallows that, which silently took the usage frame down with it. Verified on the real stack, not just in unit tests. The e2e mock answered `stream_options: {include_usage: true}` with nothing at all, so every streamed response looked usage-less and the whole recorder path went unexercised; it now reports a real prompt count that grows each round, the way a provider does. A spawned turn then tracks the server's own number live — 15k -> 18k -> 21k -> 24k, +3k per round trip — and readings are asserted never to dip, which is what a subagent's or an aux call's window reaching the gauge would look like. The typed case moves the same way. Be clear about what that run does and does not prove: against this mock, `main` renders the identical series, because every mock round calls a tool, so tool completion and provider response coincide. The e2e establishes that the meter tracks the real count live and that nothing regressed. The behaviour that actually changes — a round reporting with no tool completion anywhere in it — is pinned at the seam instead, by a test that drives the real `record_canonical_usage` and asserts the frame reaches the wire without a single tool involved. A mock turn cannot reach that state today: it would need a round that continues without calling a tool (a `length` continuation or a reasoning-only retry), which is worth adding to the harness separately rather than smuggling into this fix. Co-authored-by: Omar Baradei <omar@kostudios.io> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
OmarB97
added a commit
that referenced
this pull request
Aug 2, 2026
#337) #332 moved the meter's sampling from tool completion to the provider's own usage, on the grounds that a round which calls no tool reports nothing at all under the old scheme. Its e2e could not show that. Every round the mock drives calls a tool, so sampling per completed tool and sampling per API response produce the same series — I said so in that PR rather than let the green tick imply more than it proved. This is the test that was missing. The turn now contains one round that names a tool the agent cannot have. That name is rejected in `conversation_loop` before the executor is ever entered (`_invalid_tool_retries`), so the round finishes a real API call, reports real usage, appends a result saying the tool does not exist, and continues — with no `tool_start`, no `tool_complete`, and therefore no orphaned tool card. It is a branch on a string, so nothing about it is timing-dependent. Three other mechanisms were considered and rejected. Malformed tool arguments never reach the executor's block path — the loop validates argument JSON first and either aborts the turn or silently re-issues the identical request, and the streaming transport repairs them earlier still. A `length` continuation works but injects a `user` message mid-turn and duplicates text into the final response. A guardrail block halts the loop by construction, which fails the "turn must continue" requirement outright. Two supporting changes: - **The mock counted tool results across the whole conversation, not the turn.** A second turn in the same chat therefore started already at the cap and collapsed to a single round — so the existing "still tracks a session the user typed" test was passing on turn-boundary events, not the mid-turn frames it was written to check. Anchoring the count to the last user message fixes it: that test goes from 2 readings to 16. Only this spec has multi-turn chats, so no other spec that shares the mock changes behaviour. - **The probe's served prompt count is exported rather than hardcoded.** The usage schedule is a function of conversation length, so a literal in the spec would drift the moment the system prompt or tool list changes size. The expected value is formatted through the status bar's own `compactNumber`, because the gauge is what the user actually reads. The evidence, which is the only reason this test is worth having: with e38eacb 15k -> 18k -> 21k -> 24k (t=3.0 / 4.4 / 5.6 / 6.0s) with it reverted FAILS — 18k never appears 18k is the round that completed no tool. Reverting only the three source files of #332 and re-running turns the new test red on exactly that assertion, and restoring them turns it green again. A test that passes either way would prove nothing, so that check is the point of the change, not a footnote to it. Co-authored-by: Omar Baradei <omar@kostudios.io> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The status-bar meter still lags through a long agentic turn. Observed
2026-08-02: it sat at 55.8k while the server was serving that same session at
65.7k+ across in-turn tool rounds.
#316 made a completed TOOL push a
token.usageframe, which is what got thegauge moving mid-turn at all. But a tool completion is the wrong signal for the
quantity being published. It reports the request that produced that tool call,
before its result was appended, so the reading is a full round trip behind
whatever is in flight — and a round that ends without calling a tool (a
reasoning-only round, a
lengthcontinuation, the final round of every turn)says nothing at all. When several of those land in a row the gauge does not
drift, it sits still, which is what "pinned" looked like.
That last step is an argument from the code, not a reproduction: the harness
cannot currently drive a turn that keeps going without calling a tool, so the
field symptom itself was not reproduced here. What is reproduced, and asserted,
is that occupancy now reaches the wire on the response rather than on the tool —
see the verification note at the end for exactly where the line is drawn.
Usage now goes out from
record_canonical_usage, the one place that sees theprovider's own prompt count land on the compressor. That is one frame per
usage-bearing API response, on the real number rather than an estimate, and
because both the chat loop and the codex app-server runtime account through it,
both are covered by the same three lines. #316's tool-completion sample stays as
a cheap second one — with de-duplication it costs nothing, and it still covers a
runtime that accounts outside that recorder.
The agent loop had offered a hook for exactly this since the live context bar
was added, and nothing ever assigned it, so the branch was dead. It was also
offered at the wrong moment — before the request, carrying
estimate_messages_tokens_rough. That estimate assumes ~4 chars/token whiledense tool-call/JSON transcripts tokenize nearer ~3.4, so it reads materially
low against the server: the over-reservation note in
model_metadatameasuredest 58,039 against server >=65,797 on deepseek-v4-flash-w2, a gap the same shape
and size as the one reported here. Publishing it would have put a number on the
meter that under-reads the window by ~13%. The estimate still does its real job
(sizing compression decisions); it just no longer pretends to be a measurement,
and the helper is renamed to
_raise_preflight_context_estimateto stop thename promising otherwise.
Three details that matter more than the wiring:
delegate_toolforwards ~30 constructor kwargs from parent to child, so akwarg is precisely how a subagent would inherit this and start reporting its
own much smaller window on the parent's gauge. Auxiliary calls — goal judge,
title, compaction — never reach the recorder at all; they account through
record_aux_usage, which touches neither the compressor nor the sessioncounters. Both isolations are asserted, the goal judge through the real
auxiliary path with its recorded row checked, so the test cannot pass by the
call simply never happening.
batch reads the same occupancy, so an unchanged gauge is dropped outright and
costs nothing. A moved gauge waits out the floor; dropping one is safe because
occupancy only climbs within a turn, so a later frame supersedes it and
message.completestill carries the turn's final number. A compaction isexempt — it is the one legitimate fall, and it is the drop the client's
monotonic guard is watching
compressionsto authorize.tool.completeis the oneemit in that callback not wrapped in
try/except; a tool result that fails toserialize raises through it and the tool executor swallows that, which
silently took the usage frame down with it.
Verified on the real stack, not just in unit tests. The e2e mock answered
stream_options: {include_usage: true}with nothing at all, so every streamedresponse looked usage-less and the whole recorder path went unexercised; it now
reports a real prompt count that grows each round, the way a provider does. A
spawned turn then tracks the server's own number live — 15k -> 18k -> 21k ->
24k, +3k per round trip — and readings are asserted never to dip, which is what
a subagent's or an aux call's window reaching the gauge would look like. The
typed case moves the same way.
Be clear about what that run does and does not prove: against this mock,
mainrenders the identical series, because every mock round calls a tool, so tool
completion and provider response coincide. The e2e establishes that the meter
tracks the real count live and that nothing regressed. The behaviour that
actually changes — a round reporting with no tool completion anywhere in it — is
pinned at the seam instead, by a test that drives the real
record_canonical_usageand asserts the frame reaches the wire without a single tool involved. A mock
turn cannot reach that state today: it would need a round that continues without
calling a tool (a
lengthcontinuation or a reasoning-only retry), which isworth adding to the harness separately rather than smuggling into this fix.