Skip to content

fix(desktop): move the context meter onto the provider's real token count - #332

Merged
OmarB97 merged 1 commit into
mainfrom
fix/context-meter-real-token-count-fork-2026-08-02
Aug 2, 2026
Merged

fix(desktop): move the context meter onto the provider's real token count#332
OmarB97 merged 1 commit into
mainfrom
fix/context-meter-real-token-count-fork-2026-08-02

Conversation

@OmarB97

@OmarB97 OmarB97 commented Aug 2, 2026

Copy link
Copy Markdown
Owner

The status-bar meter still lags through a long agentic turn. Observed
2026-08-02: it sat at 55.8k while the server was serving that same session at
65.7k+ across in-turn tool rounds.

#316 made a completed TOOL push a token.usage frame, which is what got the
gauge moving mid-turn at all. But a tool completion is the wrong signal for the
quantity being published. It reports the request that produced that tool call,
before its result was appended, so the reading is a full round trip behind
whatever is in flight — and a round that ends without calling a tool (a
reasoning-only round, a length continuation, the final round of every turn)
says nothing at all. When several of those land in a row the gauge does not
drift, it sits still, which is what "pinned" looked like.

That last step is an argument from the code, not a reproduction: the harness
cannot currently drive a turn that keeps going without calling a tool, so the
field symptom itself was not reproduced here. What is reproduced, and asserted,
is that occupancy now reaches the wire on the response rather than on the tool —
see the verification note at the end for exactly where the line is drawn.

Usage now goes out from record_canonical_usage, the one place that sees the
provider's own prompt count land on the compressor. That is one frame per
usage-bearing API response, on the real number rather than an estimate, and
because both the chat loop and the codex app-server runtime account through it,
both are covered by the same three lines. #316's tool-completion sample stays as
a cheap second one — with de-duplication it costs nothing, and it still covers a
runtime that accounts outside that recorder.

The agent loop had offered a hook for exactly this since the live context bar
was added, and nothing ever assigned it, so the branch was dead. It was also
offered at the wrong moment — before the request, carrying
estimate_messages_tokens_rough. That estimate assumes ~4 chars/token while
dense tool-call/JSON transcripts tokenize nearer ~3.4, so it reads materially
low against the server: the over-reservation note in model_metadata measured
est 58,039 against server >=65,797 on deepseek-v4-flash-w2, a gap the same shape
and size as the one reported here. Publishing it would have put a number on the
meter that under-reads the window by ~13%. The estimate still does its real job
(sizing compression decisions); it just no longer pretends to be a measurement,
and the helper is renamed to _raise_preflight_context_estimate to stop the
name promising otherwise.

Three details that matter more than the wiring:

  • The hook is a post-construction attribute, never an AIAgent kwarg.
    delegate_tool forwards ~30 constructor kwargs from parent to child, so a
    kwarg is precisely how a subagent would inherit this and start reporting its
    own much smaller window on the parent's gauge. Auxiliary calls — goal judge,
    title, compaction — never reach the recorder at all; they account through
    record_aux_usage, which touches neither the compressor nor the session
    counters. Both isolations are asserted, the goal judge through the real
    auxiliary path with its recorded row checked, so the test cannot pass by the
    call simply never happening.
  • Frames are de-duplicated and floored at 1/s. Every tool in a parallel
    batch reads the same occupancy, so an unchanged gauge is dropped outright and
    costs nothing. A moved gauge waits out the floor; dropping one is safe because
    occupancy only climbs within a turn, so a later frame supersedes it and
    message.complete still carries the turn's final number. A compaction is
    exempt — it is the one legitimate fall, and it is the drop the client's
    monotonic guard is watching compressions to authorize.
  • The usage frame now precedes the tool card. tool.complete is the one
    emit in that callback not wrapped in try/except; a tool result that fails to
    serialize raises through it and the tool executor swallows that, which
    silently took the usage frame down with it.

Verified on the real stack, not just in unit tests. The e2e mock answered
stream_options: {include_usage: true} with nothing at all, so every streamed
response looked usage-less and the whole recorder path went unexercised; it now
reports a real prompt count that grows each round, the way a provider does. A
spawned turn then tracks the server's own number live — 15k -> 18k -> 21k ->
24k, +3k per round trip — and readings are asserted never to dip, which is what
a subagent's or an aux call's window reaching the gauge would look like. The
typed case moves the same way.

Be clear about what that run does and does not prove: against this mock, main
renders the identical series, because every mock round calls a tool, so tool
completion and provider response coincide. The e2e establishes that the meter
tracks the real count live and that nothing regressed. The behaviour that
actually changes — a round reporting with no tool completion anywhere in it — is
pinned at the seam instead, by a test that drives the real record_canonical_usage
and asserts the frame reaches the wire without a single tool involved. A mock
turn cannot reach that state today: it would need a round that continues without
calling a tool (a length continuation or a reasoning-only retry), which is
worth adding to the harness separately rather than smuggling into this fix.

…ount

The status-bar meter still lags through a long agentic turn. Observed
2026-08-02: it sat at 55.8k while the server was serving that same session at
65.7k+ across in-turn tool rounds.

#316 made a completed TOOL push a `token.usage` frame, which is what got the
gauge moving mid-turn at all. But a tool completion is the wrong signal for the
quantity being published. It reports the request that *produced* that tool call,
before its result was appended, so the reading is a full round trip behind
whatever is in flight — and a round that ends without calling a tool (a
reasoning-only round, a `length` continuation, the final round of every turn)
says nothing at all. When several of those land in a row the gauge does not
drift, it sits still, which is what "pinned" looked like.

That last step is an argument from the code, not a reproduction: the harness
cannot currently drive a turn that keeps going without calling a tool, so the
field symptom itself was not reproduced here. What is reproduced, and asserted,
is that occupancy now reaches the wire on the response rather than on the tool —
see the verification note at the end for exactly where the line is drawn.

Usage now goes out from `record_canonical_usage`, the one place that sees the
provider's own prompt count land on the compressor. That is one frame per
usage-bearing API response, on the real number rather than an estimate, and
because both the chat loop and the codex app-server runtime account through it,
both are covered by the same three lines. #316's tool-completion sample stays as
a cheap second one — with de-duplication it costs nothing, and it still covers a
runtime that accounts outside that recorder.

The agent loop had offered a hook for exactly this since the live context bar
was added, and nothing ever assigned it, so the branch was dead. It was also
offered at the wrong moment — *before* the request, carrying
`estimate_messages_tokens_rough`. That estimate assumes ~4 chars/token while
dense tool-call/JSON transcripts tokenize nearer ~3.4, so it reads materially
low against the server: the over-reservation note in `model_metadata` measured
est 58,039 against server >=65,797 on deepseek-v4-flash-w2, a gap the same shape
and size as the one reported here. Publishing it would have put a number on the
meter that under-reads the window by ~13%. The estimate still does its real job
(sizing compression decisions); it just no longer pretends to be a measurement,
and the helper is renamed to `_raise_preflight_context_estimate` to stop the
name promising otherwise.

Three details that matter more than the wiring:

- **The hook is a post-construction attribute, never an AIAgent kwarg.**
  `delegate_tool` forwards ~30 constructor kwargs from parent to child, so a
  kwarg is precisely how a subagent would inherit this and start reporting its
  own much smaller window on the parent's gauge. Auxiliary calls — goal judge,
  title, compaction — never reach the recorder at all; they account through
  `record_aux_usage`, which touches neither the compressor nor the session
  counters. Both isolations are asserted, the goal judge through the real
  auxiliary path with its recorded row checked, so the test cannot pass by the
  call simply never happening.
- **Frames are de-duplicated and floored at 1/s.** Every tool in a parallel
  batch reads the same occupancy, so an unchanged gauge is dropped outright and
  costs nothing. A moved gauge waits out the floor; dropping one is safe because
  occupancy only climbs within a turn, so a later frame supersedes it and
  `message.complete` still carries the turn's final number. A compaction is
  exempt — it is the one legitimate fall, and it is the drop the client's
  monotonic guard is watching `compressions` to authorize.
- **The usage frame now precedes the tool card.** `tool.complete` is the one
  emit in that callback not wrapped in `try/except`; a tool result that fails to
  serialize raises through it and the tool executor swallows that, which
  silently took the usage frame down with it.

Verified on the real stack, not just in unit tests. The e2e mock answered
`stream_options: {include_usage: true}` with nothing at all, so every streamed
response looked usage-less and the whole recorder path went unexercised; it now
reports a real prompt count that grows each round, the way a provider does. A
spawned turn then tracks the server's own number live — 15k -> 18k -> 21k ->
24k, +3k per round trip — and readings are asserted never to dip, which is what
a subagent's or an aux call's window reaching the gauge would look like. The
typed case moves the same way.

Be clear about what that run does and does not prove: against this mock, `main`
renders the identical series, because every mock round calls a tool, so tool
completion and provider response coincide. The e2e establishes that the meter
tracks the real count live and that nothing regressed. The behaviour that
actually changes — a round reporting with no tool completion anywhere in it — is
pinned at the seam instead, by a test that drives the real `record_canonical_usage`
and asserts the frame reaches the wire without a single tool involved. A mock
turn cannot reach that state today: it would need a round that continues without
calling a tool (a `length` continuation or a reasoning-only retry), which is
worth adding to the harness separately rather than smuggling into this fix.

Co-authored-by: Omar Baradei <omar@kostudios.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@OmarB97
OmarB97 merged commit e38eacb into main Aug 2, 2026
29 of 30 checks passed
OmarB97 added a commit that referenced this pull request Aug 2, 2026
#337)

#332 moved the meter's sampling from tool completion to the provider's own
usage, on the grounds that a round which calls no tool reports nothing at all
under the old scheme. Its e2e could not show that. Every round the mock drives
calls a tool, so sampling per completed tool and sampling per API response
produce the same series — I said so in that PR rather than let the green tick
imply more than it proved. This is the test that was missing.

The turn now contains one round that names a tool the agent cannot have. That
name is rejected in `conversation_loop` before the executor is ever entered
(`_invalid_tool_retries`), so the round finishes a real API call, reports real
usage, appends a result saying the tool does not exist, and continues — with no
`tool_start`, no `tool_complete`, and therefore no orphaned tool card. It is a
branch on a string, so nothing about it is timing-dependent.

Three other mechanisms were considered and rejected. Malformed tool arguments
never reach the executor's block path — the loop validates argument JSON first
and either aborts the turn or silently re-issues the identical request, and the
streaming transport repairs them earlier still. A `length` continuation works
but injects a `user` message mid-turn and duplicates text into the final
response. A guardrail block halts the loop by construction, which fails the
"turn must continue" requirement outright.

Two supporting changes:

- **The mock counted tool results across the whole conversation, not the turn.**
  A second turn in the same chat therefore started already at the cap and
  collapsed to a single round — so the existing "still tracks a session the user
  typed" test was passing on turn-boundary events, not the mid-turn frames it
  was written to check. Anchoring the count to the last user message fixes it:
  that test goes from 2 readings to 16. Only this spec has multi-turn chats, so
  no other spec that shares the mock changes behaviour.
- **The probe's served prompt count is exported rather than hardcoded.** The
  usage schedule is a function of conversation length, so a literal in the spec
  would drift the moment the system prompt or tool list changes size. The
  expected value is formatted through the status bar's own `compactNumber`,
  because the gauge is what the user actually reads.

The evidence, which is the only reason this test is worth having:

    with e38eacb      15k -> 18k -> 21k -> 24k   (t=3.0 / 4.4 / 5.6 / 6.0s)
    with it reverted     FAILS — 18k never appears

18k is the round that completed no tool. Reverting only the three source files
of #332 and re-running turns the new test red on exactly that assertion, and
restoring them turns it green again. A test that passes either way would prove
nothing, so that check is the point of the change, not a footnote to it.

Co-authored-by: Omar Baradei <omar@kostudios.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@OmarB97
OmarB97 deleted the fix/context-meter-real-token-count-fork-2026-08-02 branch August 2, 2026 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant