[Frontend] Add stateless /v1/responses/render endpoint

[anurag6569201]

Purpose

Add a stateless POST /v1/responses/render endpoint using the same prompt rendering as /v1/responses.

A scale-out renderer is vLLM's GPU-less preprocessing service. It applies model-specific formatting, tokenization, multimodal processing, tool metadata, and sampling configuration, then returns a GenerateRequest for a separate token-in generation service. This endpoint is useful for llm-d, Dynamo, and custom disaggregated frontends.

Ordinary /v1/responses serving does not require or call this endpoint. It continues to render in-process and proceed directly to inference.

This endpoint follows the scale-out gate merged in vllm-project#54579. On a standard vllm serve, /v1/responses/render is registered only when VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1; the explicit vllm launch render service continues to expose it unless scale-out endpoints are explicitly disabled. This PR adds no separately exposed route outside that shared gate.

This PR:

• moves deterministic Responses rendering into OnlineRenderer.render_responses;
• adds /v1/responses/render to the scale-out render service;
• keeps response state, generation, streaming, and output parsing in OpenAIServingResponses;
• rejects previous_response_id at the stateless boundary;
• enforces request chat-template trust in the shared renderer; and
• preserves current-main Harmony tool-choice and Cohere-format behavior.

For integrations such as vllm-agentic-api, the gateway authorizes and rehydrates stored response state first. It can then call /v1/responses directly or use /v1/responses/render as part of an llm-d-style disaggregated deployment.

For multimodal requests, the rendered GenerateRequest contains model-processed payloads that can be substantially larger than the source media. Callers must forward that payload unchanged and provision transport/message limits and memory accordingly.

Relationship to vllm-project#38084

This implements the same endpoint intent as vllm-project#38084 against the current serving architecture. Unlike the older implementation, it uses the OnlineRenderer/ServingRender split, keeps state out of the renderer, validates request-template trust, and supports Harmony tool metadata in render-only processes.

This PR does not claim to supersede vllm-project#38084; it provides a current-main implementation for review.

Testing

• Focused pytest suites for Responses serving, render parity, the stateless render endpoint, MCP/tool setup, CLI defaults, and scale-out gating: 134 passed, 1 expected xfail
• Pre-commit hooks across all PR files: passed
• pre-commit run mypy-3.12 --all-files --hook-stage manual: passed
• git diff --check: passed

I also manually smoke-tested request pass-through with llm-d on a Kubernetes deployment running on NVIDIA DGX Spark, using gpt-oss-20b and Qwen/Qwen3-0.6B. This validated request handoff only; it was not a performance, quality, or model evaluation.

Broader device-backed coverage will run in CI. Model evaluations were not run because this changes frontend rendering boundaries rather than model execution, weights, kernels, or sampling behavior.

AI assistance

This change was developed with assistance from OpenAI Codex and Claude Code/Fable. The change has undergone automated and adversarial review. The human submitter reviewed every changed line, reran the relevant checks, and confirmed end-to-end understanding.

Source merge-base: 52358e6e192aeae73dd3764046c98fc4b156ea83
Source head: a1c3cbc2a4f0f243cc49453d9c1a4e2adbea71ff

[shipwright-agent]

⚠️ Shipwright · Approve with conditions

Recommendation: approve PR #4 with conditions · Tier T3
Checks: 0 total · 0 needing attention

Next step: an authorized approver must satisfy the approval condition.

Findings (5)

• HIGH The render endpoint is stateless and rejects previous_response_id, but render_responses accepts previous_messages and previous_response_outputs as separate optional arguments. · docs/serving/online_serving/renderer.md:28
  • Fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright.
• HIGH render_responses_request returns GenerateRequest with token_offsets=engine_input.get("prompt_token_offsets") and features=self._extract_mm_features(engine_input). · vllm/entrypoints/scale_out/render/serving.py:237
  • Fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright.
• HIGH render_responses_harmony_messages applies tok_params.apply_post_tokenization to a TokensPrompt and then reads prompt["prompt_token_ids"]. · vllm/renderers/online_renderer.py:269
  • Fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright.
• HIGH The new /v1/responses/render endpoint is documented as requiring Bearer token auth when --api-key is configured, but the diff only updates docs/usage/security.md. · docs/usage/security.md:173
  • Fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright.
• LOW The new endpoint returns NotImplementedError when the model does not support Responses Render API, but the route is registered unconditionally. · vllm/entrypoints/scale_out/render/api_router.py:96
  • Fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright.

Conditions

• human approval required (T3): apply the approval label

Fireworks usage: 46,001 input · 707 output · 46,708 total tokens · $0.0106 · 12s · 0 fix iteration(s)

Open the Shipwright check for full evidence and the audit bundle. Use /shipwright rerun to verify again.

[shipwright-agent]

Shipwright · HIGH

The render endpoint is stateless and rejects previous_response_id, but render_responses accepts previous_messages and previous_response_outputs as separate optional arguments.

Impact: The render endpoint is stateless and rejects previous_response_id, but render_responses accepts previous_messages and previous_response_outputs as separate optional arguments. The public endpoint always passes None for both, so the inline-history support described in docs is not actually reachable through /v1/responses/render. A caller reading the docs will expect inline history to work, but the endpoint only suppor…

Suggested fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright.

[shipwright-agent]

Shipwright · HIGH

The new /v1/responses/render endpoint is documented as requiring Bearer token auth when --api-key is configured, but the diff only updates docs/usage/security.md.

Impact: The new /v1/responses/render endpoint is documented as requiring Bearer token auth when --api-key is configured, but the diff only updates docs/usage/security.md. No code change in vllm/entrypoints/scale_out/render/api_router.py adds an explicit auth dependency. If the scale-out render router is mounted without the same auth middleware as the main /v1 router, this endpoint could be reachable without authentication.…

Suggested fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright.