feat(cli+tui): provider quota and rate-limit in status bar - #53375
feat(cli+tui): provider quota and rate-limit in status bar#53375rafaumeu wants to merge 1 commit into
Conversation
tonydwb
left a comment
There was a problem hiding this comment.
Code Review Summary
Verdict: Approved
Good addition of provider quota and rate-limit display in the status bar. The new _fetch_zai_account_usage function follows the established pattern for other providers (OpenRouter, etc.) with proper error handling and timeout.
Looks Good
- Follows established pattern for provider usage fetching
- Proper timeout and error handling
- No security concerns
- Single-concern feature addition
Reviewed by Hermes Agent
01c57b1 to
5e3f4fa
Compare
Multi-provider rate-limit supportResearch findingsTested all 8 providers in our config with real API calls. Results:
What was done (3rd commit in this PR)Since Groq, Mistral, Cerebras and SambaNova don't have dedicated quota/usage endpoints but do send rate-limit headers on chat completions, the right approach is to make the existing
Tested with real headersAll 5 provider formats parse correctly:
NVIDIA, Google, and Cloudflare don't expose rate-limit data at all — they will show nothing in the status bar (no API to call). |
- Add Z.AI, Cloudflare, Google, NVIDIA quota fetchers in account_usage.py - Add quota/cost segment in CLI status bar with TTL cache (60s) - Add quota/cost fields in TUI gateway usage payload - Dual-source: fetch_account_usage primary, rate_limit_tracker fallback - Timezone-safe reset timer using datetime.now(timezone.utc) Closes NousResearch#53306
fef7ecc to
bd70f5a
Compare
teknium1
left a comment
There was a problem hiding this comment.
Thanks for extending the existing account-usage and rate-limit work. The gap is real: current main only surfaces these details through /usage (cli.py:9647-9710), not the status bars.
Problems
- The TUI portion is payload-only:
tui_gateway/server.py:3171adds fields, but the PR changes noui-tui/files. CurrentUsagehas no quota fields (ui-tui/src/types.ts:170-184) andStatusRuledoes not render them (ui-tui/src/components/appChrome.tsx:426-440). - The CLI renderer synchronously calls the fetcher through
cli.py:4702; the new fetchers use 10-secondhttpx.Clientcalls (for exampleagent/account_usage.py:687-692). This conflicts with #53306's requirement that render paths avoid network I/O. - The cache is process-global rather than provider/session keyed (
cli.py:52,tui_gateway/server.py:3063), so one session can receive another session's quota. The TUI also retries empty results every usage update because{}is falsy attui_gateway/server.py:3063. - The new helpers infer provider from
agent.model(cli.py:57,tui_gateway/server.py:3068) instead of the canonicalagent.providerused by current/usage(cli.py:9691).
Suggested changes
- Refresh keyed snapshots asynchronously and keep rendering read-only.
- Use
agent.provider, add the Ink contract/rendering, and cover parser, cache, and UI behavior with tests.
Automated hermes-sweeper review.
| """Fetch provider quota with a TTL cache to avoid blocking the render loop.""" | ||
| global _quota_cache, _quota_cache_ts | ||
| now = time.monotonic() | ||
| if _quota_cache is not None and (now - _quota_cache_ts) < QUOTA_CACHE_TTL: |
There was a problem hiding this comment.
This cache is process-global and has no provider, credential, or session key. A second agent rendered within 60 seconds will receive the first agent's quota snapshot. Cache per active provider/account identity instead.
| result: Dict[str, Any] = {} | ||
| try: | ||
| from agent.account_usage import fetch_account_usage | ||
| _provider = (getattr(agent, "model", None) or "").split("/")[0] |
There was a problem hiding this comment.
Use agent.provider as the primary input here. The established /usage path does that at current cli.py:9691; deriving it from a model string drops provider identity whenever the model lacks a provider/model prefix.
| """Cached quota fetch for TUI gateway — same TTL pattern as CLI.""" | ||
| global _tui_quota_cache, _tui_quota_cache_ts | ||
| now = time.monotonic() | ||
| if _tui_quota_cache and (now - _tui_quota_cache_ts) < TUI_QUOTA_CACHE_TTL: |
There was a problem hiding this comment.
An unavailable provider produces {} at line 3107, but {} is falsy, so this guard refetches on every _get_usage invocation instead of observing the TTL. Test cache initialization separately from cache contents.
| pass | ||
| # --- quota / cost (cached) --- | ||
| try: | ||
| usage.update(_get_tui_quota_snapshot(agent)) |
There was a problem hiding this comment.
This only adds fields to the backend payload. The PR changes no ui-tui/ files; current ui-tui/src/types.ts has no quota fields and ui-tui/src/components/appChrome.tsx does not render them, so the advertised TUI status-bar segment remains invisible.
Summary
Closes #53306
Adds provider quota/cost tracking to both CLI and TUI status bars with a dual-source approach:
Changes
agent/account_usage.py (+87 lines)
_fetch_zai_account_usage()— fetches Z.AI (Zhipu) token quota via the monitoring endpoint/api/monitor/usage/quota/limitfetch_account_usage(): when the provider name is not directly recognised (e.g.xai-oauthhittingapi.z.ai), resolves tozaiby inspecting the host. Falls back tocustom:zaifor API key resolution.cli.py (+46 lines)
_get_status_bar_snapshot()using dual-source pattern:fetch_account_usage()as primary (queries provider quota APIs)rate_limit_tracker.format_rate_limit_compact()as fallback (parses rate-limit headers from last response)datetime.now(timezone.utc)and forcestimezone.utcon naivereset_atvalues, so the countdown is correct regardless of system timezone (tested across UTC-3, UTC, UTC+9).tui_gateway/server.py (+41 lines)
_get_usage()for the TUI WebSocket payload (quota_pct,quota_reset,quota_rl_textfields).Design decisions
try/except— failure is silent, status bar simply omits the quota segment.rate_limit_trackerfallback ensures providers without a quota API still show useful info.base_urlhost get resolved automatically.Tested
TOKENS_LIMITandTIME_LIMITwithpercentageandnextResetTime(epoch ms).ast.parse) on all 3 files.