Skip to content

feat(gateway): emit token_usage events after each API call for real-time context bar - #38822

Closed
OmarB97 wants to merge 2 commits into
NousResearch:mainfrom
OmarB97:token-usage-events-20260604
Closed

feat(gateway): emit token_usage events after each API call for real-time context bar#38822
OmarB97 wants to merge 2 commits into
NousResearch:mainfrom
OmarB97:token-usage-events-20260604

Conversation

@OmarB97

@OmarB97 OmarB97 commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Problem

The desktop app's context window progress bar only updates at the end of a turn, causing a jarring jump. Token count data is only sent to the UI via session snapshots that fire after the turn completes.

Solution

After every API call that updates session token counters in conversation_loop.py, emit a structured token.usage event through the existing status_callback_status_update → WebSocket event pipe. The desktop app's @nous-research/ui context bar component can then subscribe to token.usage events and update smoothly during a turn.

Changes

File Change
run_agent.py Add _emit_token_usage() method — JSON-encodes token counts + context dimensions, fires via status_callback
agent/conversation_loop.py Call _emit_token_usage() after session token counters are updated
tui_gateway/server.py Handle kind=token_usage in _status_update — parse JSON payload, re-emit as typed token.usage event

Event Payload

{
  "input_tokens": 12345,
  "output_tokens": 678,
  "total_tokens": 13023,
  "context_tokens": 54321,
  "context_length": 131072,
  "context_pct": 41.5
}

Frontend Note

The @nous-research/ui npm package (v0.18.2) renders the context bar component. A separate change in that package is needed to subscribe to token.usage events. This PR provides the backend event pipe — the frontend half is independent.

Testing

  • Fire-and-forget via status_callback — never blocks the conversation loop
  • Graceful degradation: if no status_callback is wired, the event is silently dropped
  • JSON parse errors in the gateway are swallowed
  • Token counters still updated via the existing path (no regression)

Omar B added 2 commits May 31, 2026 12:54
…ime context bar

After every API call that updates session token counters, emit a structured
token.usage event through the existing status_callback → _status_update →
WebSocket event pipe. This gives the desktop app's context bar component
(@nous-research/ui) the raw material to animate smoothly during a turn
instead of jumping to the final value at end-of-turn.

Changes:
- run_agent.py: add _emit_token_usage() method that JSON-encodes
  token counts and context dimensions, then fires via status_callback
- conversation_loop.py: call _emit_token_usage() after session_*_tokens +=
- tui_gateway/server.py: handle kind=token_usage in _status_update by
  parsing the JSON payload and re-emitting as a typed 'token.usage' event

Token.usage payload:
  { input_tokens, output_tokens, total_tokens, context_tokens,
    context_length, context_pct }

The frontend side (@nous-research/ui npm package) requires a separate
upstream change to subscribe to 'token.usage' events for the context bar.
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/tui Terminal UI (ui-tui/ + tui_gateway/) labels Jun 4, 2026
@OmarB97

OmarB97 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #39370, which carries the full token-usage pipeline (_emit_token_usage, in-loop emission incl. preflight estimates, gateway token.usage bridge) rebased on current main; the bundled TUI fallback-provider fix is tracked in #36056. Closing to reduce reviewer surface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/gateway Gateway runner, session dispatch, delivery comp/tui Terminal UI (ui-tui/ + tui_gateway/) P3 Low — cosmetic, nice to have type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants