[franciscojavierarceo]

Overview

active upstream review: vllm-project/vllm#63902. this fork PR keeps the isolated native-stack diff and previous comments. all layers were rebased onto 3d81a510f9; the fork's pinned base was advanced to that commit.

add stateless, non-streaming POST /v1/responses through the existing vllm-chat pipeline, with input replay, function tools, reasoning, and usage.

upstream review order: 1/3: renderer vllm-project/vllm#63901 → 2/3: non-streaming vllm-project/vllm#63902 → 3/3: streaming vllm-project/vllm#63903. all three upstream review PRs target main; later diffs include earlier layers.

this PR includes and depends on vllm-project#63901. the fork layer shows its isolated diff.

Claims

• lower Responses input and function-call/result replay through the shared chat path, reusing compatible chat/audio/tool primitives.
• validate request controls and preserve model-resolved sampling defaults.
• use the current frontend's typed engine failures and numeric HTTP errors, with real route coverage for 400/503 failures and unknown-response 404s.
• reject stream: true with HTTP 400 on param: stream until layer 3.

Validation

Latest upstream synchronization — October 28

current head: d5b7df30aa; rebased onto upstream f4fd1ab68e. the fork keeps its native stacked bases; upstream PRs still target main.

• resolved the shared error-handler overlap with upstream's missing-LoRA 404 change. both NotFound and ResponseNotFound retain their existing wire envelopes; the LoRA/gRPC and Responses HTTP regressions pass.
• cargo test --locked -p vllm-server -p vllm-text --lib -- --test-threads=1: 579 server + 121 text passed, one existing text test ignored.
• strict Clippy for server/chat/text/CLI, applicable pre-commit hooks, and git diff --check: passed. cargo nextest remains unavailable.
• range-diff shows only the error-handler conflict adaptation; all other feature/fix patches are unchanged. all branch tips passed merge-tree checks against f4fd1ab68e.
• the old umbrella [Rust Frontend] Add OpenAI Responses API endpoint vllm-project/vllm#58783 was synchronized without rewriting its original history. its source tree equals the tested complete stack and it stays draft.

the model-backed HTTP/Codex evidence below predates this synchronization (complete-stack head 3db4c9abf0); it was not repeated after this rebase. today's evidence covers compilation and deterministic wire/route/renderer regressions, not new model qualification. CI reruns on these new heads are separate from the earlier successful workflows. agentic-api was not changed or pushed.

Earlier review and runtime evidence

reviewed and revised on October 27; non-streaming head: 91a782aa08.

• a regression first reproduced the bug: token-limited responses had overall incomplete status but a final message item marked completed.
• this layer's cargo test --locked -p vllm-server -p vllm-text --lib -- --test-threads=1: 552 server + 118 text passed, one existing text test ignored.
• added a route regression for reasoning followed by a truncated function call: earlier output stays completed; only the final message/function item becomes incomplete.
• strict Clippy, applicable pre-commit hooks, formatting, and git diff --check: passed.
• all three tips merge cleanly with upstream f8f7634cb4; the fork's isolated stack bases are preserved.

commands ran from rust/ with OPENSSL_DIR pointing to the local OpenSSL build. cargo nextest was unavailable. full-stack tests ran serially because of the previously reproduced shared chat audio-metrics collector collision.

fresh model-backed HTTP and real Codex checks passed on the revised complete stack, as recorded in vllm-project#63903. this intermediate branch was tested separately with the in-process HTTP route harness; it does not contain streaming.

Details

Responses owns its wire envelopes and conversions while reusing the chat generation path and matching shared audio/tool primitives. the rebase preserves upstream's other route registrations and error serialization rather than restoring the older envelope.

• omitted or null stream selects non-streaming. omitted/null watermarking and explicit overrides follow current Chat Completions semantics.
• metadata is a string map limited to 16 pairs, 64-character keys, and 512-character values.
• omitted temperature/top-p use model defaults; explicit overrides are preserved and checked against the engine request.
• unknown service tiers, invalid summary/verbosity values, unsupported include outputs, and nonzero top_logprobs are rejected.
• unknown previous_response_id uses the current shared NotFoundError envelope: HTTP 404, param: response_id, numeric code: 404.

the frontend remains stateless: clients replay history, and agentic-api owns persistence and tool execution. the shared native adapter tracked by vllm-project/agentic-api#359 and vllm-project/agentic-api#382 is still unfinished. accepted reasoning-summary, text-verbosity, and encrypted-reasoning inputs are documented compatibility no-ops; valid service tiers are echoed without scheduling guarantees. persistence, retrieval/cancellation, built-in/custom/namespace tool execution, Harmony rendering, output logprobs, and allowed_tools are out of scope. the rendered Responses documentation describes the supported subset.

these are the existing vllm-project#58783 layers opened upstream at @BugenZhao's request, rather than competing implementations. the original authored history and discussion remain in vllm-project#58783. @gcanlin's vllm-project#53380 implementation is credited in the endpoint and streaming commits. vllm-project#56124 covers render endpoints and vllm-project#51535 covers chat allowed_tools; neither duplicates this endpoint.

this layer is 13 files, +2987 / -1 relative to vllm-project#63901. tests stay under the test-only route module. agentic-api compatibility follow-ups remain local and unpushed. Codex assistance was used; human review of every changed line and independent validation are required before merge.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

[Copilot]

Copilot review overview

🟡 Changes recommended

Valid controls are silently ignored and some response metadata misrepresents actual behavior.

Review effort: Balanced
Findings: 7

Open (7)

• Error code format differs from Python vLLM contract · New
• Requested include outputs are silently ignored · New
• Response reports incorrect resolved sampling parameters · New
• Ignored reasoning summary and text verbosity controls · New
• Metadata accepts invalid shapes and exceeds documented limits · New
• Invalid service tiers are accepted and echoed · New
• Nullable stream value is rejected during deserialization · New

What changed in this PR

Adds a stateless, non-streaming OpenAI-compatible Responses API to the Rust frontend.

Changes:

• Adds request validation, chat lowering, response assembly, and tool/reasoning replay.
• Registers create, retrieve, and cancel routes with runtime offloading.
• Adds unit and HTTP regression coverage.

File	Description
rust/​src/​server/​src/​routes/​tests/​responses.rs	Adds Responses HTTP tests.
rust/​src/​server/​src/​routes/​tests.rs	Registers the test module.
rust/​src/​server/​src/​routes/​openai/​responses/​validate.rs	Validates supported request features.
rust/​src/​server/​src/​routes/​openai/​responses/​types.rs	Defines request and response wire types.
rust/​src/​server/​src/​routes/​openai/​responses/​convert.rs	Converts Responses requests and outputs.
rust/​src/​server/​src/​routes/​openai/​responses.rs	Implements endpoint handlers.
rust/​src/​server/​src/​routes/​openai/​mod.rs	Exports Responses handlers.
rust/​src/​server/​src/​routes.rs	Registers HTTP routes.
rust/​src/​server/​src/​middleware/​offload.rs	Offloads response generation work.
rust/​src/​server/​src/​error.rs	Adds unknown-response errors.
rust/​src/​parser/​src/​tool/​mod.rs	Allows omitted function schemas.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.