feat: Add client-side rate limiter for Hermes Agent - #13307
Closed
LVT382009 wants to merge 1 commit into
Closed
Conversation
Implements sliding window rate limiting to prevent 429 errors from API providers. Core features: - SlidingWindowRateLimiter: Thread-safe sliding window algorithm - ProviderRateLimiterRegistry: Per-provider rate limits with fallback - Zero-cost NOOP: No overhead when disabled (rpm=0) - Built-in plugin: /ratelimit slash commands for runtime control Configuration: - HERMES_REQUESTS_PER_MINUTE env var - rate_limit.requests_per_minute in config.yaml - Per-provider limits (nvidia, openai, mistral, openrouter, groq) Integration: - run_agent.py: acquire() before every API call - hermes_cli/config.py: Configuration support - plugins/rate-limiter/: Runtime control surface Tested with: - 30 unit tests (100% pass rate) - Real NVIDIA API (34/35 successful, no 429 errors) - Sequential requests (rate limiter blocks correctly) Resolves: Rate limiting for API providers
13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements sliding window rate limiting to prevent 429 errors from API providers.
Core Features
/ratelimitslash commands for runtime controlConfiguration
HERMES_REQUESTS_PER_MINUTEenvironment variablerate_limit.requests_per_minutein config.yamlIntegration
run_agent.py:acquire()before every API callhermes_cli/config.py: Configuration supportplugins/rate-limiter/: Runtime control surfaceTested With
Usage
What does this PR do?
This PR adds a client-side rate limiter for Hermes Agent that paces outgoing API calls to prevent hitting provider rate limits (429 errors). The implementation uses a sliding window algorithm that tracks requests in a 60-second window and blocks when the limit is reached.
Problem Solved: Users frequently hit 429 errors when making rapid API calls to providers like NVIDIA, OpenAI, and others. This causes failed requests and poor user experience.
Why This Approach:
Related Issue
Resolves rate limiting for API providers to prevent 429 errors.
Type of Change
Changes Made
agent/rate_limiter.py(new): Core rate limiter implementation with SlidingWindowRateLimiter and ProviderRateLimiterRegistry classesplugins/rate-limiter/__init__.py(new): Plugin registration with register() function and session hooksplugins/rate-limiter/commands.py(new): Slash command handlers (help, status, enable, disable, set)plugins/rate-limiter/plugin.yaml(new): Plugin metadata (provides_slash_commands: ratelimit)run_agent.py(modified): Integration at lines 94 (import), 1469 (registry initialization), 5868 (non-streaming API call), 6230 (streaming API call)hermes_cli/config.py(modified): Configuration support with rate_limit section and HERMES_REQUESTS_PER_MINUTE env varHow to Test
Unit Tests:
python3 -m pytest tests/test_rate_limiter.py -v # Expected: 30/30 tests passEnable Rate Limiter:
Test Rate Limiting:
Test Plugin Commands:
Verify Zero Overhead When Disabled:
Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/AFor New Skills
N/A - This is a built-in plugin, not a skill.
Screenshots / Logs
Unit Test Results
Real NVIDIA API Test Results
Rate Limiter Debug Output