feat: rate limit escalation circuit breaker

[bobrykov]

No description provided.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 18a17ff0-fb2c-4301-972e-1443165e0e47

📥 Commits

Reviewing files that changed from the base of the PR and between 0e38c02 and 5645572.

📒 Files selected for processing (2)

• rustfmt.toml
• src/stream/handler.rs

💤 Files with no reviewable changes (1)

• rustfmt.toml

🚧 Files skipped from review as they are similar to previous changes (1)

• src/stream/handler.rs

---

📝 Walkthrough

Walkthrough

Rate-limit handling now uses structured errors, deadline-clamped retries, explicit escalation variants, and fallback circuit-breaker routing. Stream event processing was refactored into helpers, with tests covering retry decisions, timeout behavior, error mapping, and fallback activation.

Changes

Rate-limit escalation

Layer / File(s)	Summary
Rate-limit error contracts src/api/error.rs, src/error.rs, src/stream/handler.rs	Adds structured rate-limit constructors and escalation variants with retry metadata, recoverability, formatting, and documentation.
Deadline-aware stream retry flow src/stream/handler.rs	Adds per-call rate-limit retry decisions, cancellable deadline-clamped sleeps, and helper-based event polling and application.
Error mapping and fallback routing src/engine/bare/stream.rs, src/engine/bare.rs	Maps stream escalation into LoopError and routes escalated failures to model-failure circuit-breaker accounting.
Behavior validation and release notes src/stream/handler.rs, src/engine/bare.rs, src/engine/bare/stream.rs, src/api/error.rs, src/error.rs, CHANGELOG.md	Adds coverage for retry thresholds, deadline clamping, mappings, fallback activation, and documents the new behavior.

Sequence Diagram(s)

sequenceDiagram
  participant ApiClient
  participant StreamHandler
  participant BareLoop
  participant FallbackManager
  ApiClient-->>StreamHandler: rate-limited stream outcome
  StreamHandler->>StreamHandler: retry with bounded backoff
  StreamHandler-->>BareLoop: RateLimitEscalation
  BareLoop->>FallbackManager: record_model_failure()
  FallbackManager-->>BareLoop: fallback mode

Loading

Possibly related PRs

• dch-labs/loopctl#19: Introduces the BareLoop streaming path extended by this escalation handling.
• dch-labs/loopctl#29: Establishes the stream-handler wiring and error conversion extended here.
• dch-labs/loopctl#43: Also modifies BareLoop::do_stream error control flow.

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: rate-limit escalation now triggers a circuit breaker.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 50.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/retry-after

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/stream/handler.rs`:
- Around line 1411-1418: Update the event polling logic around `event_deadline`
and the `tokio::select!` in the stream handler to compute the earliest
applicable deadline: use `initial_event_timeout` when `events_processed == 0`,
otherwise `per_event_timeout`, and cap it by the overall `total_stream_timeout`
deadline. Add the total-deadline branch to `tokio::select!` and return the
appropriate timeout error when either deadline expires.
- Around line 1206-1229: Rate-limit retries currently consume the outer generic
retry budget, causing escalation thresholds to be reached too early. Update the
stream retry loop around rate_limit_retry and the outer attempt handling to
track transport retries separately, allowing RateLimitConfig and
fallback_after_retries to govern rate-limit attempts independently while
preserving existing transport retry limits.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e4d6142d-a9dc-476f-9b9d-4f3259706021

📥 Commits

Reviewing files that changed from the base of the PR and between 85e2185 and 33851a3.

📒 Files selected for processing (6)

• CHANGELOG.md
• src/api/error.rs
• src/engine/bare.rs
• src/engine/bare/stream.rs
• src/error.rs
• src/stream/handler.rs