feat(gateway): add per-turn token usage footer to gateway responses - #17592
Open
hyofeng wants to merge 4 commits into
Open
feat(gateway): add per-turn token usage footer to gateway responses#17592hyofeng wants to merge 4 commits into
hyofeng wants to merge 4 commits into
Conversation
- Add token usage snapshot/diff utilities in gateway/run.py - Append inline-code usage line (t=/i=/o=) after every gateway reply - Support cached tokens and reasoning tokens display - Remove stale vercel_runtime env-var mapping
alt-glitch
added
type/feature
Collaborator
Move token-usage rendering from inline run.py logic into gateway/runtime_footer.py as a new 'tokens' field, aligning with the merged NousResearch#17026 runtime-footer architecture. Changes: - runtime_footer.py: add _render_token_usage(), _format_usage_int(); support 'tokens' field in format_runtime_footer() and build_footer_line() - run.py: keep _snapshot_agent_token_usage() / _diff_agent_token_usage() for per-turn delta tracking; pass token_usage through build_footer_line(); add token_usage to approval-path result dicts Usage: set display.runtime_footer.fields to include 'tokens' e.g. fields: [model, context_pct, cwd, tokens]
agent.run_conversation() return value does not include input/output
token counters. Snapshot agent session counters before and after
the run to compute accurate per-turn deltas.
Previous fallback (result.get('input_tokens')) always returned 0.
- Strip agent's self-reported usage:t=... line before appending gateway runtime footer to avoid duplicate token numbers in responses - Wrap runtime footer in backticks for consistent inline-code formatting
This was referenced May 16, 2026
teknium1
reviewed
Jul 12, 2026
teknium1
left a comment
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for adapting the proposal to the existing runtime-footer surface.
Problems
gateway/runtime_footer.py:178wraps every existing footer in backticks, not just token data. Current main has exact plain-text formatter contracts intests/gateway/test_runtime_footer.py:70,:95,:127,:137, and:147; this diff does not update that coverage.- The new
tokensfield remains outside the default field list, while the footer remains disabled by default. That is compatible with the opt-in design from #17026, but it does not match the PR's “every reply” description.website/docs/user-guide/configuration.md:1483also lists onlymodel,context_pct, andcwd. - Current main's proxy branch (
gateway/run.py:16899-16910) returns no token counters (gateway/run.py:16795-16806), so local snapshots alone cannot provide the proposed value for every gateway response.
Suggested changes
- Preserve the existing formatter contract or update its exact-output tests deliberately.
- Add token rendering, per-turn delta, streaming, and proxy-mode coverage; document
tokensas an opt-in field. - Reapply the result plumbing to current
gateway/run.py::_run_agent_inner, whose local result assembly is now atgateway/run.py:18789-19064.
Automated hermes-sweeper review.
| if not parts: | ||
| return "" | ||
| return _SEP.join(parts) | ||
| return f"`{_SEP.join(parts)}`" |
Contributor
There was a problem hiding this comment.
This changes all pre-existing runtime-footer output to inline code, not only the new token field. Current main has exact plain-text output contracts in tests/gateway/test_runtime_footer.py (for example line 70); preserve that contract or update the corresponding tests and documentation deliberately.
| rel = _home_relative_cwd(cwd or os.environ.get("TERMINAL_CWD", "")) | ||
| if rel: | ||
| parts.append(rel) | ||
| elif field == "tokens": |
Contributor
There was a problem hiding this comment.
tokens is opt-in because _DEFAULT_FIELDS remains unchanged, but this PR adds no configuration documentation or renderer tests for the new field. Please document the required fields: [..., tokens] configuration and cover token-only, mixed, zero, cache, and reasoning cases.
| _approval_session_key = session_key or "" | ||
| _approval_session_token = set_current_session_key(_approval_session_key) | ||
| register_gateway_notify(_approval_session_key, _approval_notify_sync) | ||
| _usage_before = _snapshot_agent_token_usage(agent) |
Contributor
There was a problem hiding this comment.
This snapshots only the local AIAgent path. The gateway also has a proxy execution branch on current main (_run_agent_inner → _run_agent_via_proxy) that returns no counters, so define whether proxy responses propagate remote usage or intentionally omit the tokens field before claiming coverage for all gateway replies.
teknium1
added
sweeper:risk-session-state
vanhci
pushed a commit
to vanhci/hermes-agent
that referenced
this pull request
Aug 11, 2026
…fields Extend display.runtime_footer with token-count and reasoning-effort fields: - tokens_in / tokens_out: compact k-formatted (15.9k) per-turn token usage - effort: agent reasoning effort label from agent.reasoning_effort - context_pct now renders as 'ctx N%' (aligns with cc-connect style) - Feishu renders the footer as a card-style quote block (divider + '> ') instead of plain inline text All new fields are opt-in: the default field set is unchanged. Related: NousResearch#19922, NousResearch#17592, NousResearch#64526, NousResearch#28978
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add automatic per-turn token usage reporting to all gateway responses. Every agent reply now ends with a compact inline-code usage footer.
Changes
_snapshot_agent_token_usage(),_diff_agent_token_usage(),_build_gateway_usage_line()utilities for tracking and formatting token usageusage:t=<total> i=<input> o=<output>footer after every gateway response(+ N c)) and reasoning tokens ((r N)) displayvercel_runtimeenv-var mappingTest Plan