Skip to content

feat(gateway): display model name and context usage in response footer - #16800

Closed
RichardQidian wants to merge 1 commit into
NousResearch:mainfrom
RichardQidian:feat/context-usage-display
Closed

feat(gateway): display model name and context usage in response footer#16800
RichardQidian wants to merge 1 commit into
NousResearch:mainfrom
RichardQidian:feat/context-usage-display

Conversation

@RichardQidian

Copy link
Copy Markdown

📊 Display Model Name & Context Usage

This PR adds an automatic context usage footer to non-streaming gateway responses, helping users monitor their context window usage in real-time.

What it shows

Every non-streaming response now includes a footer like:

---
🤖 anthropic/claude-sonnet-4  |  📊 Context: 45,230 / 200,000 (23%)

When context usage reaches 70%+, it also shows a warning:

---
🤖 anthropic/claude-sonnet-4  |  📊 Context: 148,500 / 200,000 (74%) ⚠️ 建议 /new 开启新会话

Changes

  • Add context usage footer to non-streaming responses
  • Track context_length in agent result (previously only last_prompt_tokens was tracked)
  • Warning shown when context usage exceeds 70%, suggesting user to start a new session

Why this matters

  • Users often lose track of context consumption in long conversations
  • When context gets full, model behavior degrades (truncation, lost instructions)
  • Gives users visibility and actionable guidance before hitting the wall
  • Helps users choose appropriate models for their context needs

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery labels Apr 28, 2026
Add automatic context usage footer to non-streaming responses showing:
- Model name being used
- Current context token count vs total context length
- Percentage of context used
- Warning when context usage >= 70% suggesting /new

This helps users monitor context window usage and know when to start
a fresh session.

Closes: improves user experience for long-running conversations
@RichardQidian
RichardQidian force-pushed the feat/context-usage-display branch from ddb15d6 to e2695dd Compare May 4, 2026 02:42
@zjy282

zjy282 commented May 5, 2026

Copy link
Copy Markdown

+1 — would love to see this merged. Switching models often via gateway (Discord/Telegram) makes it hard to know which model is replying; a per-response footer with the active model is exactly the visibility that is missing today. Context usage % is a great bonus.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the contribution, @RichardQidian! This feature landed on main via a parallel implementation in PR #17026 and shipped in v2026.4.30.


This is an automated hermes-sweeper review.

  • gateway/runtime_footer.py (commit e123f4ecf) implements build_footer_line() / format_runtime_footer() / resolve_footer_config() — model name, context percentage, and cwd in a compact footer.
  • gateway/run.py lines 6283–6302 and 6513–6523 wire the footer into both streaming and non-streaming response paths.
  • The shipped version is opt-in (display.runtime_footer.enabled: true) with per-platform overrides, a /footer on|off toggle command, and a configurable fields list — a superset of what this PR added.

Closing as implemented. If you'd like to contribute further (e.g. adjusting the default-on threshold or adding the warning text), feel free to open a new targeted PR against current main.

@teknium1 teknium1 closed this Jun 10, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have sweeper:implemented-on-main Sweeper: behavior already present on current main type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants