feat(gateway): derive backend request ids from rid and the middleware request id

[slin1237]

Description

Problem

rid passthrough existed on generate/embeddings/classify/rerank but not chat or completions (#1874) — and on gRPC it was only honored by /generate: embeddings/classify minted embed-/classify- ids ignoring their own rid fields. Worse, every request-building stage minted its own id independently of the request-id middleware (middleware/request_id.rs), so three ids existed per request — the middleware id in gateway logs, the stage-minted id in engine logs and the response body, and whatever the client sent — none correlated.

Solution

One resolver, every path. helpers::resolve_request_id is now the single source of backend request ids for all gRPC request-building stages (chat, harmony chat/responses, messages, completions incl. batch, generate, embeddings, classify):

1. Protocol rid (added to chat, completions, and messages; already on generate/embeddings/classify) — used verbatim outside PD; per-attempt -{uuid} suffix in PD (matching /generate's long-standing NIXL-lease rule).
2. Else the middleware request id — client-sent via the configured headers (x-request-id, x-correlation-id, …) or middleware-generated; carried into the pipeline on the tenant request metadata (tenant resolution runs inside RequestIdLayer and copies the extension — zero router-signature changes). Always suffixed -{uuid} per execution, which keeps retries, PD attempts, and responses tool-loop iterations unique while staying grep-able by prefix.
3. Else the endpoint-prefixed mint, as before (fixture/test contexts).

HTTP mode needs no mechanism: the typed rid fields serialize through natively, and the middleware already echoes x-request-id.

Net effect: gateway log id, engine log id, and response body id are one lineage for every request — rid when given, the middleware id otherwise. Batched completions keep the clean shared id on the response and suffix per-sub engine ids.

Behavior notes:

• Response body ids for default traffic become {middleware-id}-{uuid} (e.g. chatcmpl-Abc…-0198…); endpoint prefixes are preserved, including a new msg_ arm in the middleware generator so Anthropic message ids keep their shape.
• The middleware's path matching switched from ordered contains scans to ends_with suffix compares — sub-resource paths (e.g. GET /v1/responses/{id}) now generate the generic req- log id instead of resp-; those ids never reach response bodies.
• gRPC responses tool-loop iterations now log under {request-id}-{uuid} per iteration, correlating engine work back to the originating request.

Changes

• crates/protocols: rid: Option<String> on ChatCompletionRequest, CompletionRequest, CreateMessageRequest (typed field replaces the other-flatten passthrough on HTTP).
• grpc/common/stages/helpers.rs: resolve_request_id + middleware_request_id; RequestType::rid() accessor in context.rs.
• All six request-building stages route through the resolver; batch completions compose {shared}-p{i} with the same uniqueness rules.
• middleware/tenant_resolution.rs: carries RequestId on the request meta; middleware/request_id.rs: msg_ arm + ends_with matching.
• Go SDK: optional Rid on ChatCompletionRequest.
• Docs: backend-id derivation note in the logging guide.

Test Plan

• Unit: resolver priority/suffix rules (4 tests), middleware prefix mapping, tenant-meta carries RequestId.
• cargo test -p smg -p openai-protocol -p smg-grpc-client: all green (smg lib 1233; api 106, routing 94, spec 96, security 50, …; protocols 84+126; grpc_client 65) — full-server api_tests exercise the new id lineage end-to-end.
• e2e: chat rid → response.id == rid; chat x-request-id: corr-abc → response.id starts with corr-abc-; completions scalar + batch rid → response id equals rid.
• cargo clippy --all-targets -- -D warnings, cargo +nightly fmt --check, ruff clean; Go SDK package builds (go build .; examples need libsmg_go, unchanged).

Checklist

• cargo +nightly fmt passes
• cargo clippy --all-targets --all-features -- -D warnings passes
• (Optional) Documentation updated
• (Optional) Please join us on Slack #sig-smg to discuss, review, and merge PRs

Summary by CodeRabbit

• New Features
  • Added optional request ID (rid) support to chat, completion, messages, and other request types.
  • Client-provided rid is forwarded for backend log correlation and reflected in response ids.
• Documentation
  • Clarified request-correlation behavior and how engine/gateway request IDs are derived.
• Bug Fixes
  • Improved endpoint-specific request ID prefixing and middleware request ID propagation.
  • Enhanced request ID generation rules for retries, batch prompts, and disaggregated mode.
• Tests
  • Added e2e and unit tests covering request ID passthrough and response id formatting.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: dfdb6581-45e1-4edf-b0cb-ab1e52639ac4

📥 Commits

Reviewing files that changed from the base of the PR and between e48427b and 7191a7f.

📒 Files selected for processing (21)

• bindings/golang/client.go
• crates/grpc_client/src/vllm_engine.rs
• crates/protocols/src/chat.rs
• crates/protocols/src/completion.rs
• crates/protocols/src/messages.rs
• docs/getting-started/logging.md
• e2e_test/chat_completions/test_openai_server.py
• e2e_test/completions/test_basic.py
• model_gateway/benches/request_processing.rs
• model_gateway/src/middleware/request_id.rs
• model_gateway/src/middleware/tenant_resolution.rs
• model_gateway/src/routers/grpc/common/stages/helpers.rs
• model_gateway/src/routers/grpc/context.rs
• model_gateway/src/routers/grpc/harmony/stages/request_building.rs
• model_gateway/src/routers/grpc/regular/stages/chat/request_building.rs
• model_gateway/src/routers/grpc/regular/stages/completion/request_building.rs
• model_gateway/src/routers/grpc/regular/stages/embedding/request_building.rs
• model_gateway/src/routers/grpc/regular/stages/generate/request_building.rs
• model_gateway/src/routers/grpc/regular/stages/messages/request_building.rs
• model_gateway/src/routers/grpc/utils/message_utils.rs
• model_gateway/tests/routing/test_openai_routing.rs

---

📝 Walkthrough

Walkthrough

Adds optional rid fields to request protocols and the Go client, preserves middleware request IDs, centralizes backend request-ID resolution, updates gRPC request-building stages for disaggregated execution, and adds documentation plus unit and end-to-end coverage.

Changes

Request ID propagation

Layer / File(s)	Summary
Request ID request contracts crates/protocols/src/{chat.rs,completion.rs,messages.rs}, bindings/golang/client.go, request helpers and fixtures	Chat, completion, and message requests now expose optional rid values; constructors and benchmark fixtures initialize the field explicitly.
Middleware and shared ID resolution model_gateway/src/middleware/*, model_gateway/src/routers/grpc/{context.rs,common/stages/helpers.rs}	Middleware request IDs are retained in route metadata, endpoint prefixes use suffix matching, and shared resolution selects protocol, middleware, or generated IDs.
Backend request-building integration model_gateway/src/routers/grpc/{harmony,regular}/stages/**	Harmony and regular gRPC stages use shared request-ID resolution with endpoint prefixes and disaggregated-mode uniqueness handling.
End-to-end behavior and documentation docs/getting-started/logging.md, e2e_test/{chat_completions,completions}/*	Documentation and OpenAI tests cover request-ID propagation, response IDs, batch IDs, and header prefixes.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant GatewayMiddleware
  participant RequestType
  participant RequestBuilder
  participant Backend
  Client->>GatewayMiddleware: Send rid or x-request-id
  GatewayMiddleware->>RequestType: Preserve request metadata
  RequestType->>RequestBuilder: Provide protocol rid
  RequestBuilder->>Backend: Send resolved backend request ID
  Backend-->>Client: Return response id

Loading

Possibly related issues

• [Feature]: Passthrough for "rid" for chat completions #1874 — Directly covers chat rid passthrough and log-correlation behavior implemented here.

Possibly related PRs

• lightseekorg/smg#1957 — Overlaps the completion request-building path changed for request-ID resolution.
• lightseekorg/smg#744 — Overlaps Messages request-building and its msg_ request-ID prefix.
• lightseekorg/smg#915 — Overlaps the completion request-building pipeline.

Suggested labels: documentation, grpc, tests, protocols, model-gateway

Suggested reviewers: key4ng, catherinesue

Poem

A rabbit hops where request IDs flow,
Through chat and batches, neat in a row.
Middleware carries each tiny thread,
Backend logs know where they’re led.
“Rid,” says the bunny, “correlate instead!”

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: backend request IDs are derived from protocol rid and middleware request IDs.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/unified-rid

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"

---

_{Comment @coderabbitai help to get the list of available commands.}

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/getting-started/logging.md`:
- Around line 359-364: Update the request-ID documentation to distinguish the
default gRPC `{request-id}-{uuid}` behavior from protocol-level `rid` handling:
outside PD mode, use `rid` verbatim, except batch completions return the shared
`rid` while dispatching each prompt with a per-prompt ID such as `{rid}-p{i}`.
State when response IDs match the shared or per-request ID rather than implying
the rule applies universally.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 850dc0b5-1734-4373-8083-5732154f294b

📥 Commits

Reviewing files that changed from the base of the PR and between 2a11cc7 and e48427b.

📒 Files selected for processing (21)

• bindings/golang/client.go
• crates/grpc_client/src/vllm_engine.rs
• crates/protocols/src/chat.rs
• crates/protocols/src/completion.rs
• crates/protocols/src/messages.rs
• docs/getting-started/logging.md
• e2e_test/chat_completions/test_openai_server.py
• e2e_test/completions/test_basic.py
• model_gateway/benches/request_processing.rs
• model_gateway/src/middleware/request_id.rs
• model_gateway/src/middleware/tenant_resolution.rs
• model_gateway/src/routers/grpc/common/stages/helpers.rs
• model_gateway/src/routers/grpc/context.rs
• model_gateway/src/routers/grpc/harmony/stages/request_building.rs
• model_gateway/src/routers/grpc/regular/stages/chat/request_building.rs
• model_gateway/src/routers/grpc/regular/stages/completion/request_building.rs
• model_gateway/src/routers/grpc/regular/stages/embedding/request_building.rs
• model_gateway/src/routers/grpc/regular/stages/generate/request_building.rs
• model_gateway/src/routers/grpc/regular/stages/messages/request_building.rs
• model_gateway/src/routers/grpc/utils/message_utils.rs
• model_gateway/tests/routing/test_openai_routing.rs