[sherlockwu]

Stacked on #42674 (gRPC ChatComplete). That PR is stacked on #42673 (tonic 0.14).

Complete the router's native gRPC API: each inference RPC behaves like its HTTP route, in both unified and PD fleets.

Changes

RPC	Routed like
ChatComplete (#42674)	/v1/chat/completions
Complete	/v1/completions (new HTTP route too)
Generate, TextGenerate	/generate
Embed, TextEmbed, OpenAIEmbed	/v1/embeddings
Classify, OpenAIClassify	/v1/classify
Rerank	/v1/rerank

• OpenAI RPCs forward the HTTP route's JSON body as is.

• Typed RPCs are prepared from their fields with the same rules as their HTTP route, with no JSON round trip. Routing inputs are read straight from the proto, and the router's additions (rid, PD bootstrap, DP rank, sampling defaults) are set as fields on each leg.

• Text prompts are tokenized once, as their HTTP routes forward input_ids:

  • TextGenerate goes to the engine as Generate with return_text ([gRPC] Move runtime.v1 bindings to tonic 0.14, report OpenAI status codes, return text on request #42673), so the engine still supplies the text and the router never detokenizes.
  • TextEmbed goes as Embed, and Classify goes with ids.
  • They stay text RPCs under --disable-input-ids-forwarding, without a tokenizer, or (for TextGenerate) when return_text_in_logprobs is set.

• Shared code:

  • HTTP handlers and gRPC calls share one prepare step per Endpoint, and forwarding is generic over the engine RPC.
  • HTTP /v1/completions shares chat's preparation but never forwards input_ids.

• Not served: Tokenize, HealthCheck, model info and the admin RPCs still answer UNIMPLEMENTED. Score does too, since it has no router HTTP route.

• Review fixes: [Feat] sgl-router: Serve native gRPC ChatComplete #42674's review fixes (abort by rid, breaker classification, streaming status, router error codes) are carried to the generic forwarding, so they cover every RPC.

• Second review round:

  • A completions prompt list is a fan-out batch, so it is no longer pinned to one DP rank in PD, and token-id prompts are counted exactly.
  • When routing needs token counts, each text prompt in a batch is tokenized, as /generate does, instead of estimated from bytes. The body is still forwarded as sent.
  • Typed requests count the contract's default n.
  • Typed sampling values compare as the decimal sent (0.7, not the f32's 0.699999988).
  • Typed RPCs carry trace headers sent as gRPC metadata to the engine's trace_headers, except authorization.

Tests

• Mock tests: tests/proxy/grpc_routing.rs uses mock engines, so CI needs no GPUs. It covers:
  • OpenAI RPCs sending the HTTP body and relaying chunks;
  • PD for chat and typed Generate, under both legacy and reorg routing: the same bootstrap room on both legs, and the same rid;
  • a typed unary Embed;
  • TextGenerate/TextEmbed reaching the engine as Generate/Embed with the router's ids;
  • cancellation;
  • errors.
• Unit test: checks that the router's additions never override a caller's value.
• Real engines (manual run, not CI), with router HTTP, router gRPC and direct engine gRPC compared at temperature 0: 27/27 checks pass.
  • Unified Qwen3-4B: chat, completions, Generate, TextGenerate, streaming and non-streaming all match HTTP. The engine's Generate with return_text returns HTTP's text, and none without it. An over-long prompt gets the same 400.
  • PD (prefill + decode over NIXL): every generation RPC matches HTTP and matches the unified outputs.
  • Qwen3-Embedding-0.6B: OpenAIEmbed, Embed, TextEmbed, Classify and OpenAIClassify all match.
  • Reorg routing (--chat-routing reorg), unified and PD: 19/19 generation checks match HTTP and the unified outputs.
  • Qwen3-Reranker-0.6B: Rerank matches. The engine's own gRPC Rerank serialization fix is in [gRPC] Move runtime.v1 bindings to tonic 0.14, report OpenAI status codes, return text on request #42673.

---

CI States

Latest PR Test (Base): ✅ Run #38857817139
Latest PR Test (Extra): ❌ Run #38857817136
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.

[ShangmingCai]

Looks good.