fix(grpc): use e.message() instead of Display for tonic::Status in user-facing errors

[CatherineSue]

Description

Problem

Relates to #650.

gRPC backend errors include raw tonic::Status debug output in user-facing error messages. For example, a TRT-LLM validation error returns:

{"message": "Failed to start generation: status: 'Client specified an invalid argument', self: \"Greedy decoding...\", metadata: {\"content-type\": \"application/grpc\"}"}

This is because format!("Failed to start generation: {e}") uses tonic::Status's Display impl, which dumps the full debug representation including status:, self:, and metadata:.

Solution

Replace {e} with e.message() in all error paths where e is a tonic::Status. This produces clean messages like:

{"message": "Failed to start generation: Greedy decoding in the LLM API does not allow multiple returns..."}

Changes

• request_execution.rs — 7 call sites (generate, embed, dual dispatch, sequential PD)
• response_collection.rs — 1 call site (stream collection)
• regular/streaming.rs — 3 call sites (stream and prefill errors)
• harmony/streaming.rs — 4 call sites (stream, prefill, decode errors)

Test Plan

• cargo check -p smg passes
• Pre-commit (rustfmt, clippy) passes

Checklist

• cargo +nightly fmt passes
• cargo clippy --all-targets --all-features -- -D warnings passes
• (Optional) Documentation updated

Summary by CodeRabbit

• Bug Fixes
  • Improved error message formatting consistency across multiple system components to enhance error reporting reliability.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue where gRPC backend errors were exposing raw tonic::Status debug output in user-facing messages. By consistently utilizing the e.message() method of tonic::Status instead of its default Display implementation, the change ensures that error messages are more readable and relevant to the user, improving the overall clarity and professionalism of error reporting within the system.

Highlights

• Improved User-Facing Error Messages: Replaced the default Display implementation for tonic::Status with e.message() in error formatting, ensuring that user-facing error messages from gRPC backend calls are clean and concise, without exposing internal debug details.
• Consistent Error Handling: Applied this fix across various gRPC error paths, including request execution (generation, embedding, dual dispatch, sequential PD), response collection, and streaming processors (Harmony and Regular), standardizing the format of error messages presented to users.

Changelog

• model_gateway/src/routers/grpc/common/response_collection.rs
  • Updated error message formatting for worker stream failures to use e.message().
• model_gateway/src/routers/grpc/common/stages/request_execution.rs
  • Adjusted error message for failed generation start to use e.message().
  • Modified error message for failed embedding start to use e.message().
  • Updated error messages for prefill and decode worker startup failures in dual dispatch to use e.message().
  • Refined error message for prefill worker startup failure in sequential PD to use e.message().
  • Improved error message for prefill stream errors in sequential PD to use e.message().
  • Corrected error message for decode worker startup failure in sequential PD to use e.message().
• model_gateway/src/routers/grpc/harmony/streaming.rs
  • Updated prefill stream error message to use e.message().
  • Adjusted general stream error message to use e.message().
  • Modified prefill stream error message during cached token collection to use e.message().
  • Refined decode stream error message to use e.message().
• model_gateway/src/routers/grpc/regular/streaming.rs
  • Adjusted general stream error message in the main loop to use e.message().
  • Updated prefill stream error message during logprob collection to use e.message().
  • Modified general stream error message for completion token mapping to use e.message().
  • Refined prefill stream error message during input logprob processing to use e.message().
  • Corrected general stream error message for completion token mapping to use e.message().

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[coderabbitai]

📝 Walkthrough

Walkthrough

Error message formatting is standardized across gRPC error handling paths to explicitly call .message() on error objects instead of directly interpolating the error struct in format strings. Changes affect response collection, request execution, and streaming modules.

Changes

Cohort / File(s)	Summary
Common error handling model_gateway/src/routers/grpc/common/response_collection.rs, model_gateway/src/routers/grpc/common/stages/request_execution.rs	Error message formatting updated to use e.message() in worker failure messages across multiple execution paths (single, embed, dual dispatch, sequential prefill/decode flows).
Streaming error handling model_gateway/src/routers/grpc/harmony/streaming.rs, model_gateway/src/routers/grpc/regular/streaming.rs	Stream error messages now use e.message() for both prefill and decode stream error paths instead of direct error interpolation.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• fix: gRPC error handling — proper status codes and circuit breaker accuracy #645: Directly related to error string construction and tonic::Status message mapping used in HTTP error payloads throughout these gRPC handlers.

Suggested labels

grpc, model-gateway

Suggested reviewers

• key4ng
• slin1237

Poem

🐰 Errors now speak more clearly and bright,
No struct confusion, just messages right,
.message() calls keep the streams flowing free,
Worker failures formatted beautifully! ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Title check	✅ Passed	The title accurately describes the main change: replacing Display formatting ({e}) with e.message() for tonic::Status errors in user-facing error messages across multiple files.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch chang/grpc-error-message

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request addresses an issue where raw tonic::Status debug output was being included in user-facing gRPC error messages. The changes correctly replace the use of the Display implementation for tonic::Status with e.message() across multiple files, ensuring that only the clean error message is presented to the user. The modifications in response_collection.rs, request_execution.rs, harmony/streaming.rs, and regular/streaming.rs are consistent and effectively resolve the problem described. The code is clean and the fix is well-implemented.