fix(kv): stop retrying SubscribeKvEvents when backend returns UNIMPLEMENTED

[CatherineSue]

Description

Problem

When a gRPC backend (e.g. vLLM) does not implement the SubscribeKvEvents RPC, the KV event monitor retries forever with exponential backoff (100ms → 30s), spamming warning logs. This RPC will never succeed for backends that don't support it.

Solution

Before retrying, downcast the error to tonic::Status and check for Code::Unimplemented. If detected, log a single warning and exit the subscription loop permanently for that worker.

Changes

• In kv_event_monitor.rs subscription_loop, added early return when subscribe_kv_events fails with gRPC UNIMPLEMENTED status code.

Test Plan

• Deploy SMG against a vLLM backend without SubscribeKvEvents support
• Verify: single warning log "Backend does not implement SubscribeKvEvents, disabling KV event subscription for this worker" appears once
• Verify: no repeated retry spam in logs
• Other transient errors (e.g. connection refused) still retry with backoff as before

Checklist

• cargo +nightly fmt passes
• cargo clippy --all-targets --all-features -- -D warnings passes
• (Optional) Documentation updated

Summary by CodeRabbit

• Bug Fixes
  • Improved KV event subscription error handling to gracefully disable the feature when unavailable, preventing unnecessary retry loops. Other error conditions continue to retry with backoff as designed.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the KV event monitor's error handling by introducing a mechanism to gracefully manage situations where a backend does not support the SubscribeKvEvents RPC. Previously, such scenarios led to continuous, unsuccessful retries and excessive log output. The update ensures that the system identifies this specific error, logs it once, and then stops attempting to subscribe, thereby improving system stability and reducing log noise.

Highlights

• Error Handling Improvement: Implemented a specific check for UNIMPLEMENTED gRPC status codes when subscribing to KV events, preventing infinite retries.
• Log Spam Reduction: When a backend does not support SubscribeKvEvents, the system will now log a single warning and cease retrying, rather than spamming logs with repeated warnings.
• Targeted Fix: This change specifically addresses issues with backends like vLLM that do not implement the SubscribeKvEvents RPC.

Changelog

• model_gateway/src/core/kv_event_monitor.rs
  • Added logic to detect and handle UNIMPLEMENTED gRPC status codes from subscribe_kv_events.
  • Introduced an early return from the subscription loop if the UNIMPLEMENTED status is encountered, preventing further retries.

Activity

• The pull request was created by CatherineSue to address an issue with excessive logging and retries when a gRPC backend does not implement SubscribeKvEvents.
• The author provided a clear description of the problem, solution, and test plan, including verification steps for log behavior.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a0dc691 and 9733636.

📒 Files selected for processing (1)

• model_gateway/src/core/kv_event_monitor.rs

---

📝 Walkthrough

Walkthrough

The KV event monitor now handles Unimplemented status errors from initial subscribe_kv_events calls by logging a warning and returning early to disable subscription. Other errors continue retrying with backoff as before.

Changes

Cohort / File(s)	Summary
KV Event Subscription Error Handling model_gateway/src/core/kv_event_monitor.rs	Added explicit handling for Unimplemented gRPC status on initial KV event subscription, logging a warning and early return to gracefully disable the feature rather than entering retry loops.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

• feat(grpc): add subscribe_kv_events to all backend clients #558: Adds and exports the subscribe_kv_events client methods that this PR handles failures for.

Suggested labels

grpc, model-gateway

Suggested reviewers

• key4ng

Poem

🐰 A graceful retreat when paths aren't found,
The monitor logs and bows without a sound,
No endless loops, just wisdom to know—
When Unimplemented calls, it's time to let go! ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding a fix to stop retrying the SubscribeKvEvents RPC when the backend returns UNIMPLEMENTED status, which directly corresponds to the changeset's purpose of preventing indefinite retries for unsupported RPC methods.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch chang/fix-kv-unimplemented

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request correctly addresses an issue where the KV event monitor would endlessly retry connecting to a backend that doesn't implement the SubscribeKvEvents RPC. The change introduces a check for the UNIMPLEMENTED gRPC status code and gracefully stops the subscription loop for that worker, preventing log spam and unnecessary retries. The implementation is sound. I have one minor suggestion to make the code more concise.

[gemini-code-assist]

For improved conciseness and readability, you can combine the nested if let and if statements into a single if statement using the matches! macro with a guard. This expresses the condition in a single, fluent line.

                    if matches!(e.downcast_ref::<tonic::Status>(), Some(status) if status.code() == tonic::Code::Unimplemented) {
                        warn!(
                            worker_url = %worker_url,
                            "Backend does not implement SubscribeKvEvents, \
                             disabling KV event subscription for this worker"
                        );
                        return;
                    }