[roytman]

What type of PR is this?

/kind feature

What this PR does / why we need it:

The decode step derived its wire format from the client's path and rejected use_openai_format, so a tokens-in pipeline sent its last leg as the client's own API while encode and prefill used /inference/v1/generate. The prompt was tokenized twice: once by the renderer for prefix-cache scoring, and again by the decode worker. Decode is also where the chat template and the tool and reasoning parsers run, so keeping it on the OpenAI route tied parser behavior to the GPU worker image.

Decode now resolves its format through resolveFormat like encode and prefill, so pipeline.use_openai_format: false addresses every leg at the generate route. Three things follow from that route's contract.

output_mode asks the worker what to return beside the token IDs: text for the detokenized text, derender for the parsed assistant message (content, reasoning, tool_calls). A worker predating the field ignores it and answers with token IDs only, which carries no text for the client, so a response that does not echo the requested mode is an error rather than an answer. vLLM refuses derender together with stream: true, so that leg is sent unstreamed and a streaming client receives its answer in a single frame.

sampling_params is required on the generate route and has no server-side default. A client whose output cap sits at the top level of another API's body (max_tokens, max_completion_tokens, max_output_tokens) has it moved under sampling_params; without the move the worker falls back to its own max_tokens default of 16 and truncates the answer.

wrap_chat_completion rebuilds the chat completions envelope for a client that called /v1/chat/completions, since vLLM serves output_mode on the generate envelope rather than returning an OpenAI one (the response schema would otherwise depend on a request field, which the upstream RFC rejected). Default false. The message moves across whole, so the reasoning and tool_calls a derender response carries survive without this package modeling them.

conditional-decode still derives its format from the client's path: it probes that route with the body the client sent, so use_openai_format cannot apply to it and the step keeps rejecting the key.

The first commit is a no-behavior-change refactor: the sidecar declared choices, finish_reason, message, index, delta and the assistant role as unexported constants of its own package, and the coordinator needs the same names to build a chat completions body. They move to pkg/common/request, which already owns the request field names.

Scope and limits

• Non-streaming /v1/chat/completions only for the envelope transform. A streamed answer is a sequence of chat.completion.chunk frames this step does not assemble, so a streaming request is forwarded unchanged. Upstream streaming derender ([Frontend] Stream derender on /inference/v1/generate vllm-project/vllm#63103) is still open.
• /v1/completions, /v1/responses and /v1/messages keep the envelope their own schema declares. Those transforms belong in their own issues.
• output_mode: tokens is refused in config: decode streams its response to the client, so that mode would answer with token IDs and no text.

Required vLLM version

Both modes need a worker that loads a tokenizer; vLLM answers 400 under --tokens-only or --skip-tokenizer-init, and derender on a worker with a tool or reasoning parser configured also requires a parse_context from /render, which this pipeline does not send.

Capability	vLLM PR	First release containing it
output_mode, value text	vllm-project/vllm#58588	v0.32.0
output_mode: derender	vllm-project/vllm#60082	unreleased, in v0.33.0rc0

output_mode: text requires vLLM >= v0.32.0; output_mode: derender requires vLLM > v0.32.0 (main, or v0.33.0 once released). The e2e lane still pins an older render
er image, so it does not exercise either mode against real vLLM yet (#3620).

Test plan

Unit tests, all passing:

• Decode reads use_openai_format and rewrites a chat-completions request to the generate route with the rendered token IDs and the client's cap moved under samp ling_params
• A client already on the generate route keeps the fields it sent
• output_mode defaults to text, accepts derender, and refuses tokens and any unknown value at startup
• The chat completions envelope is rebuilt from a derender response, preserving reasoning and tool_calls, and from a text response by building the message
  from its text
• Every generate-only field is dropped from the envelope (request_id, output_mode, the KV and EC transfer params, prompt token IDs, metrics, per-choice token I
  Ds and weight versions)
• A response that does not echo the requested output_mode is an error rather than an empty answer
• The transform is skipped for a streaming request and for a client on another route
• conditional-decode still rejects use_openai_format

Verified end to end on a CoreWeave H200 cluster (Qwen3-VL-32B-Instruct, TP=2, one DisaggregatedSet, 10 pods, kv-nixl and ec-nixl):

• All three legs reach /inference/v1/generate, confirmed from the coordinator's own step logs and from the model servers' access logs
• The decode leg carries output_mode on the wire, captured at TRACE
• A multimodal /v1/chat/completions request returns a chat.completion body that the official openai Python SDK (2.48.0) parses as a typed ChatCompletion, w
  ith no generate-only fields present
• A /inference/v1/generate client still receives the generate envelope unchanged
• derender with stream: true is refused by the worker with 400, and the step's unstreamed leg avoids it

Which issue(s) this PR fixes:

Fixes #3689

Release note:

The coordinator's decode step now honors pipeline.use_openai_format: false and sends the decode leg to /inference/v1/generate, so a tokens-in pipeline tokenizes a prom
pt once instead of twice. A new decode step parameter, output_mode, selects what the worker returns beside the token IDs: text for the detokenized text (vLLM >= v0.32.
0) or derender for the parsed assistant message (vLLM > v0.32.0). A second parameter, wrap_chat_completion, rebuilds the chat completions envelope for non-streaming /v
1/chat/completions clients, which would otherwise read the generate response shape.