fix(rate-limit): honor zero token refill rate

[Moersity]

Description

Problem

rate_limit_tokens_per_second = 0 passes configuration validation and is supported by TokenBucket as a zero-refill concurrency limiter. However, startup filtered out zero and fell back to max_concurrent_requests as the refill rate. The Helm chart also omitted the CLI flag when the value was zero.

Solution

Preserve explicit non-negative refill rates when constructing the token bucket, and render the Helm CLI argument for zero. Keep negative values filtered so unchecked configurations cannot be cast to an invalid usize.

Changes

• Honor an explicit zero token refill rate in AppContextBuilder.
• Pass auth.rateLimitTokensPerSecond: 0 through the Helm chart.
• Add a regression test proving that zero disables time-based refill while returned tokens remain reusable.

Test Plan

• cargo +nightly fmt --all -- --check
• cargo clippy --all-targets --all-features -- -D warnings
• cargo test
• helm lint deploy/helm/smg
• Render the chart with auth.rateLimitTokensPerSecond=0 and verify that --rate-limit-tokens-per-second 0 is present.
• Render the chart with the default -1 and verify that the rate-limit argument is absent.

Checklist

• cargo +nightly fmt passes
• cargo clippy --all-targets --all-features -- -D warnings passes
• (Optional) Documentation updated
• (Optional) Please join us on Slack #sig-smg to discuss, review, and merge PRs

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 8c972f9c-da8b-40d4-95db-0f67f4d46ce7

📥 Commits

Reviewing files that changed from the base of the PR and between e124339 and fda958b.

📒 Files selected for processing (1)

• model_gateway/src/app_context.rs

🚧 Files skipped from review as they are similar to previous changes (1)

• model_gateway/src/app_context.rs

---

📝 Walkthrough

Summary by CodeRabbit

• Bug Fixes
  • Zero-valued rate-limit settings are now honored correctly.
  • Rate limiting no longer substitutes a concurrency limit when token replenishment is set to zero.
  • Tokens already available remain usable while replenishment is disabled.
  • Rate-limited services now consistently enforce configurations that disable token replenishment, preventing unexpected token refills while preserving access to available tokens.

Walkthrough

The rate-limit configuration now preserves an explicit zero token refill rate from Helm output through model gateway initialization. An asynchronous test verifies that zero refill prevents replenishment while initial and returned tokens remain usable.

Changes

Rate-limit configuration and runtime behavior

Layer / File(s)	Summary
Helm rate-limit argument emission deploy/helm/smg/templates/_helpers.tpl	The router emits --rate-limit-tokens-per-second when auth.rateLimitTokensPerSecond is zero or greater. Negative values remain excluded.
Gateway zero-refill behavior model_gateway/src/app_context.rs	The model gateway preserves a configured zero refill rate. The test verifies initial capacity, no replenishment, and token reuse after return.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested labels: tests

Suggested reviewers: catherinesue, key4ng

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description clearly explains the zero token refill rate fix, Helm chart update, and regression test.
Title check	✅ Passed	The title clearly and concisely identifies the main change: honoring a zero token refill rate.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@model_gateway/src/app_context.rs`:
- Around line 718-736: Strengthen the regression test
explicit_zero_rate_limit_disables_refill so the old fallback rate is guaranteed
to replenish at least one token during the wait: adjust the bucket capacity and
sleep duration accordingly while preserving the zero-rate assertions. Then run
the pr-test-analyzer agent to verify the test adequately detects fallback
behavior.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a41039b5-3493-4736-9368-c7e4acb5db99

📥 Commits

Reviewing files that changed from the base of the PR and between 416bc20 and e124339.

📒 Files selected for processing (2)

• deploy/helm/smg/templates/_helpers.tpl
• model_gateway/src/app_context.rs