Repository navigation
Rob 1942 holmes return rate limit when happen - #854
Conversation
WalkthroughAdds litellm rate-limit handling: server endpoints now catch litellm.exceptions.RateLimitError and return HTTP 429. Streaming utilities add an SSE "error" event, helpers to format SSE error messages, and catch RateLimitError in stream formatters to emit structured SSE error events instead of aborting. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Server
participant LiteLLM
Client->>Server: HTTP POST /api/(investigate|workload_*|issue_chat|chat)
Server->>LiteLLM: Request completion
alt Rate limit hit
LiteLLM-->>Server: RateLimitError
Server-->>Client: HTTP 429 {error}
else Normal
LiteLLM-->>Server: Response
Server-->>Client: 200 {result}
end
sequenceDiagram
participant Client
participant Server
participant StreamFmt as Stream Formatter
participant LiteLLM
Client->>Server: GET /api/stream/investigate (SSE)
Server->>StreamFmt: start stream
StreamFmt->>LiteLLM: stream tokens
alt Rate limit hit
LiteLLM-->>StreamFmt: RateLimitError
StreamFmt-->>Client: SSE event "error" {success:false, code:5204, msg:"Rate limit exceeded", description}
else Normal stream
LiteLLM-->>StreamFmt: tokens
StreamFmt-->>Client: SSE events (data)
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. 📜 Recent review detailsConfiguration used: CodeRabbit UI 💡 Knowledge Base configuration:
You can enable these sources in your CodeRabbit configuration. 📒 Files selected for processing (1)
🧰 Additional context used📓 Path-based instructions (1)**/*.py📄 CodeRabbit Inference Engine (CLAUDE.md)
Files:
🪛 Ruff (0.12.2)server.py152-152: Within an (B904) 243-243: Within an (B904) 274-274: Within an (B904) 303-303: Within an (B904) 375-375: Within an (B904) ⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (6)
holmes/utils/stream.py (6)
22-25: Add return type tocreate_sse_message.Project requires type hints; add the return type.
Apply this diff:
-def create_sse_message(event_type: str, data: Optional[dict] = None): +def create_sse_message(event_type: str, data: Optional[dict] = None) -> str:
28-38: Type-hintcreate_sse_error_messagereturn type.Minor typing improvement to comply with mypy.
Apply this diff:
-def create_sse_error_message(description: str, error_code: int, msg: str): +def create_sse_error_message(description: str, error_code: int, msg: str) -> str:
40-45: Optional: narrow exception import to avoid importing the fulllitellmmodule.Catching
RateLimitErrorcan be done by importing the class directly (from litellm.exceptions import RateLimitError) and usingexcept RateLimitErrorhere as well, which reduces module-level import surface.Do you want me to patch this file to import
RateLimitErrordirectly and adjust the excepts accordingly?
47-69: Add explicit return type and parameter typing to the formatter.Yield type is
str(SSE lines). Also typerunbooksto align with usage.Apply this diff:
-def stream_investigate_formatter( - call_stream: Generator[StreamMessage, None, None], runbooks -): +def stream_investigate_formatter( + call_stream: Generator[StreamMessage, None, None], + runbooks: Optional[List[dict]], +) -> Generator[str, None, None]:
75-90: Add explicit return type to chat formatter.Comply with type-hinting guidelines.
Apply this diff:
-def stream_chat_formatter( - call_stream: Generator[StreamMessage, None, None], - followups: Optional[List[dict]] = None, -): +def stream_chat_formatter( + call_stream: Generator[StreamMessage, None, None], + followups: Optional[List[dict]] = None, +) -> Generator[str, None, None]:
67-69: Consider logging rate-limit events for observability.Emitting SSE error is great for clients; a server-side warning log would aid ops visibility and rate-limit diagnostics.
Do you want me to add a
logging.warning("Rate limit hit during streaming", exc_info=True)here? I can wire in a safeimport loggingat the file top.Also applies to: 88-89
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
holmes/utils/stream.py(2 hunks)server.py(6 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit Inference Engine (CLAUDE.md)
**/*.py: ALWAYS place Python imports at the top of the file, not inside functions or methods
Type hints are required (project is type-checked with mypy)
Use Ruff for formatting and linting (configured in pyproject.toml)
Files:
server.pyholmes/utils/stream.py
🧬 Code Graph Analysis (1)
holmes/utils/stream.py (2)
holmes/core/investigation_structured_output.py (1)
process_response_into_sections(225-238)holmes/core/tool_calling_llm.py (1)
call_stream(573-708)
🪛 Ruff (0.12.2)
server.py
152-152: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
242-242: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
273-273: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
302-302: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
374-374: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Pre-commit checks
- GitHub Check: llm_evals
- GitHub Check: Pre-commit checks
🔇 Additional comments (2)
server.py (1)
138-156: Mapping rate-limit errors to HTTP 429 is the right UX.Catching
RateLimitErrorand returning 429 across these endpoints is consistent and user-friendly. Nice.Also applies to: 185-246, 248-277, 279-306, 318-377
holmes/utils/stream.py (1)
10-15: SSE ERROR event addition looks good.The explicit
StreamEvents.ERROR = "error"provides a clean, standardized path for structured streaming failures.
2 participants
No description provided.