feat(agent): pre-emptive RPM throttling using x-ratelimit response headers

[Tranquil-Flow]

What does this PR do?

Adds pre-emptive RPM throttling for Anthropic, OpenAI, OpenRouter, and Nous providers using x-ratelimit-remaining-requests response headers. When remaining requests fall to ≤ threshold (default: 2), sleeps until the minute window resets — preventing 429 errors before they happen.

Problem: Hermes already parses x-ratelimit-* headers (agent/rate_limit_tracker.py) and displays them via /usage, but never acts on them. When the agent approaches a provider's RPM limit, it burns through remaining requests and hits 429s, triggering expensive retry/failover loops. The header data is right there — we just weren't using it for pacing.

Additionally fixes a non-streaming header capture gap: _capture_rate_limits() was only called after streaming responses (line ~4597 of run_agent.py). Non-streaming API calls never captured headers, so the throttler would have no data to work with on those code paths. Non-streaming paths now also capture via .response / ._response attributes.

Architecture:

• New agent/rpm_throttler.py with maybe_throttle(state, provider, threshold=2) — checks requests_min.remaining, sleeps if ≤ threshold.
• RPM_THROTTLE_PROVIDERS frozenset: anthropic, openai, openrouter, nous.
• Sleep capped at 65s (MAX_THROTTLE_SLEEP), minimum 0.5s to avoid busy-spin.
• Elapsed time adjusted: uses remaining_seconds_now from RateLimitBucket which accounts for time since header capture.
• Integration: _maybe_rpm_throttle() method in run_agent.py wraps maybe_throttle() with exception safety; called before each LLM API call in the main agent loop (line ~7812).

Config: Currently uses hardcoded defaults (threshold=2). The threshold parameter is exposed as a function argument for future config integration (e.g., rpm_throttle_threshold in custom_providers).

Related Issue

Closes #7489

Related: Phase 1 (concurrency semaphore for z.ai/Kimi): #7479. Existing header parser: agent/rate_limit_tracker.py.

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 🔒 Security fix
• 📝 Documentation update
• ✅ Tests (adding or improving test coverage)
• ♻️ Refactor (no behavior change)
• 🎯 New skill (bundled or hub)

Changes Made

• New agent/rpm_throttler.py implementing maybe_throttle() and provider allow-list
• agent/run_agent.py: _maybe_rpm_throttle() wrapper, called before each LLM API call; non-streaming _interruptible_api_call() now captures headers from response before returning
• 20 unit tests in tests/agent/test_rpm_throttler.py

How to Test

1. Run new unit tests: pytest tests/agent/test_rpm_throttler.py -q (20 passed)
2. Full agent suite: pytest tests/agent/ -q → 1041 passed, 1 pre-existing failure (unrelated)
3. Manual: configure OpenRouter provider, monitor RPM headers via /usage, confirm throttle activates at low remaining counts

Test coverage:

• Provider filtering (throttles anthropic/openai/openrouter/nous, skips zai/ollama, case insensitive)
• No-op cases (None state, empty state, no RPM data, above threshold, near-zero reset)
• Throttle fires (at threshold, zero remaining, max sleep cap, min sleep guard)
• Custom threshold (higher threshold, threshold=0)
• Elapsed time adjustment (partial elapsed, fully elapsed)
• Logging (logs when throttling, silent when not)

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: macOS (darwin)

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

pytest tests/agent/test_rpm_throttler.py -q
20 passed

pytest tests/agent/ -q
1041 passed, 1 pre-existing failure (unrelated)

[Tranquil-Flow]

Re-ported onto current origin/main. The original PR was actually written against a version of main that already had the rate-limit capture infrastructure, so the rpm_throttler module ports verbatim — only the call-site wiring moved.

What changed in the re-port:

• agent/rpm_throttler.py — ported verbatim from the original PR. Self-contained module; imports only RateLimitState from agent.rate_limit_tracker (which exists on main with the exact fields the throttler uses: has_data, requests_min, bucket.remaining_seconds_now).
• AIAgent._maybe_rpm_throttle() — added on run_agent.py:1869 right after get_rate_limit_state.
• Call site moved to agent/conversation_loop.py:1073 (above the streaming/non-streaming fork). Single throttle per turn — no double-fire risk. The streaming + non-streaming entry points in agent/chat_completion_helpers.py do NOT throttle themselves to avoid double-firing when codex delegates from streaming to non-streaming.
• Non-streaming rate-limit capture added in agent/chat_completion_helpers.interruptible_api_call (parallel to the existing streaming capture). Extracts the underlying httpx response via .response / ._response. Without this, the throttle could only react to streaming-path response headers.

Providers gated to {anthropic, openai, openrouter, nous} — local/custom endpoints skipped (header semantics aren't guaranteed). Defaults preserved: threshold=2 remaining, cap=65s, min sleep=0.5s.

New head: dd0907df. 20/20 tests pass.

Note: this is framed as Phase 2 of the rate-limit hardening work (Phase 1 = concurrency semaphore for z.ai/Kimi in #7479, also re-ported this session — though that one's a much bigger surface).

[teknium1]

Thanks for carrying the existing header-tracking work into a proactive mechanism. The underlying gap remains on current main: capture is still streaming-only at agent/chat_completion_helpers.py:2176, with no preflight RPM throttle.

Problems

• agent/rpm_throttler.py:82 relies on a numeric reset value, but agent/rate_limit_tracker.py:85-89 parses with float(). OpenAI documents values such as 1s; that becomes 0.0, and line 83 skips the throttle.
• agent/rpm_throttler.py:30 advertises OpenRouter support, but OpenRouter documents X-RateLimit-* headers on its platform-limit 429 responses, not successful inference responses. The success-path capture cannot pre-empt it.
• agent/rpm_throttler.py:70 does not verify state.provider. Fallback changes agent.provider at agent/chat_completion_helpers.py:1409, while the captured state remains, so one supported provider can inherit another provider's low-quota wait.

Suggested changes

• Add provider-specific header/reset parsing tests, including OpenAI duration resets.
• Re-scope or redesign OpenRouter support around its documented quota endpoint.
• Match/clear captured state across provider switches and cover fallback transitions.

Automated hermes-sweeper review.

[teknium1]

RateLimitBucket currently obtains reset_seconds through float() in agent/rate_limit_tracker.py; OpenAI documents duration-valued resets such as 1s. That parses as 0.0, then line 83 skips throttling. Please parse the provider format and add a real-header regression before listing OpenAI as supported.

[teknium1]

This allow-list check does not establish that the cached state came from the active provider. Fallback activation changes agent.provider but does not clear _rate_limit_state, so supported-provider fallback A→B can sleep on A's depleted bucket. Compare normalized state.provider here or clear state on every provider transition.

[teknium1]

OpenRouter's current limits documentation says successful inference responses do not contain X-RateLimit-*; the headers are returned on platform-limit 429 responses. Because capture runs after a successful response, this entry cannot provide pre-emptive throttling. Please remove it or obtain state from OpenRouter's documented key endpoint.

[Tranquil-Flow]

Closing #7490 based on the sweeper review. Three concerns block a faithful salvage:

1. agent/rpm_throttler.py:82 parses with float() but OpenAI documents duration resets like 1s — that becomes 0.0 and line 83 skips throttling.
2. agent/rpm_throttler.py:30 advertises OpenRouter support, but OpenRouter documents X-RateLimit-* headers on platform-limit 429 responses, not successful inference. The success-path capture cannot pre-empt it.
3. The allow-list check at agent/rpm_throttler.py:70 doesn't verify state.provider matches the active provider. Fallback activations change agent.provider but don't clear _rate_limit_state, so A→B provider switching inherits A's depleted bucket.

Closing to clear the queue. A new PR that parses provider-format duration resets (with real-header tests), removes or re-scopes the OpenRouter entry against the documented quota endpoint, and matches/clears captured state on provider switches will land cleanly.