[franciscojavierarceo]

Overview

active upstream review: vllm-project/vllm#63903. this fork PR keeps the isolated native-stack diff and previous comments. all layers were rebased onto 3d81a510f9; the fork's pinned base was advanced to that commit.

add Responses SSE streaming with text, reasoning, function-call events, stable item IDs, and terminal handling for completion, token limits, and failures.

upstream review order: 1/3: renderer vllm-project/vllm#63901 → 2/3: non-streaming vllm-project/vllm#63902 → 3/3: streaming vllm-project/vllm#63903. all three upstream review PRs target main; later diffs include earlier layers.

this PR includes and depends on vllm-project#63901 and vllm-project#63902. the fork layer isolates the streaming changes.

Claims

• emit ordered response/item/content events, using IDs assigned by vllm-chat.
• terminate token limits with response.incomplete; close started items before response.failed, retain partial output, and preserve engine-supplied usage.
• use the existing SSE response helper and keep-alive configuration, with test-only HTTP regressions.

Validation

Latest upstream synchronization — October 28

current head: bfe6223420; rebased onto upstream f4fd1ab68e. the fork keeps its native stacked bases; upstream PRs still target main.

• resolved the shared error-handler overlap with upstream's missing-LoRA 404 change. both NotFound and ResponseNotFound retain their existing wire envelopes; the LoRA/gRPC and Responses HTTP regressions pass.
• cargo test --locked -p vllm-server -p vllm-chat -p vllm-text --lib -- --test-threads=1: 586 server + 438 chat + 121 text passed (1,145 total), one existing text test ignored.
• strict Clippy for server/chat/text/CLI, applicable pre-commit hooks, and git diff --check: passed. cargo nextest remains unavailable.
• range-diff shows only the error-handler conflict adaptation; all other feature/fix patches are unchanged. all branch tips passed merge-tree checks against f4fd1ab68e.
• the old umbrella [Rust Frontend] Add OpenAI Responses API endpoint vllm-project/vllm#58783 was synchronized without rewriting its original history. its source tree equals the tested complete stack and it stays draft.

the model-backed HTTP/Codex evidence below predates this synchronization (complete-stack head 3db4c9abf0); it was not repeated after this rebase. today's evidence covers compilation and deterministic wire/route/renderer regressions, not new model qualification. CI reruns on these new heads are separate from the earlier successful workflows. agentic-api was not changed or pushed.

Earlier review and runtime evidence

reviewed and revised on October 27; streaming head: 3db4c9abf0.

• cargo test --locked -p vllm-server -p vllm-chat -p vllm-text --lib -- --test-threads=1: 559 server + 430 chat + 118 text passed (1,107 total), one existing text test ignored.
• cargo clippy --locked -p vllm-server -p vllm-chat -p vllm-text -p vllm-cmd --all-targets -- -D warnings: passed.
• cargo build --locked -p vllm-cmd, applicable pre-commit hooks, formatting, and git diff --check: passed.
• all three tips merge cleanly with upstream f8f7634cb4; the fork's isolated stack bases are preserved.

the review found that token-limit response status was correct but the last output item was still marked completed. the final output_item.done now waits until the finish reason is known. text and function-call route regressions assert that item-done snapshots exactly equal terminal response.output, including incomplete status; earlier items stay completed.

fresh live tests, October 27, on the revised stack:

• 6/6 HTTP checks passed: JSON and streaming through direct Rust, agentic-api pass-through, and agentic-api stored execution. each token-limited response has the correct terminal status/reason and final-item status; streaming item-done snapshots equal terminal output.
• real Codex CLI 0.160.0 passed 2/2 shell-tool/replay runs through agentic-api: pass-through and forced store: true via a loopback test adapter. each executed cat nonce.txt, submitted its function-call output, returned the exact nonce, and emitted turn.completed. the adapter recorded two stored requests, with the second replaying one tool output.
• the first stored test exposed a gateway bug: its item-status enum rejected incomplete, dropping JSON items or rewriting streamed items to completed. local agentic-api commit 28665d7 fixes the shared ingestion/status conversions and custom-tool completion. 787 core + 9 stream-validation tests passed, five existing ignores; strict Clippy and all applicable pre-commit/file-size hooks passed. this follow-up remains local and unpushed, separately from feat: add shared vLLM Responses event contract vllm-project/agentic-api#382.

configuration: Qwen/Qwen3.8-27B-FP8 revision 017b9c7af6b5689d5dd426a76e0bc077eb5ca20a, NVIDIA GB10, 32k context, language-model-only, 3 GiB KV cache, Qwen3 reasoning/Qwen3 Coder tool parsers. Rust was rebuilt from the revised sources; the installed Python engine was 90531daf6, not this PR's Python checkout. the pass-through Codex run used agentic-api 8a1e4ab; the stored run used the revised 28665d7 source. Codex used an isolated text-only catalog, Responses transport, and read-only sandbox; native custom tools, apps/plugins, images, web search, and multi-agent features were disabled.

these runs establish the tested HTTP interoperability, not native library integration, model-quality/throughput qualification, unrestricted Codex support, or multimodal coverage. other model families, parallel tool calls, and Codex abort/disconnect remain unvalidated. the shared native adapter is still unfinished.

commands ran from rust/ with OPENSSL_DIR pointing to the local OpenSSL build. cargo nextest was unavailable. tests ran serially after the previously reproduced shared chat audio-metrics collector collision. upstream pre-commit workflows passed on all three heads; the two newly pushed docs builds were still pending at the last check. local checkout and GitHub docs discovery checks passed; a fresh preview-search agent could not access the preview, while manual preview retrieval confirmed the supported-subset documentation.

Details

this removes layer 2's temporary streaming rejection. the rebase consumes upstream's boxed transfer metadata without cloning, accepts the updated terminal-event fields, and handles typed error finishes.

the decode-failure regression checks partial text and stable item IDs, exactly one item close before one response.failed, no completion event, and no fabricated missing usage. abort retains the Python-compatible completed event with cancelled response status; it is separate from the token-limit incomplete event.

the frontend remains stateless: clients replay history, and agentic-api owns persistence and tool execution. the shared native adapter tracked by vllm-project/agentic-api#359 and vllm-project/agentic-api#382 is still unfinished. accepted reasoning-summary, text-verbosity, and encrypted-reasoning inputs are documented compatibility no-ops; valid service tiers are echoed without scheduling guarantees. layer 2 describes the remaining request/feature boundaries. the rendered Responses documentation now includes SSE and terminal behavior.

these are the existing vllm-project#58783 layers opened upstream at @BugenZhao's request, rather than competing implementations. the original authored history and discussion remain in vllm-project#58783. @gcanlin's vllm-project#53380 implementation is credited in the endpoint and streaming commits. this stack carries the prior implementation plus the current-main compatibility resolutions; the old source tips are backed up locally.

this layer is 4 files, +1267 / -57 relative to vllm-project#63902. Codex assistance was used. human review of every changed line and independent validation remain required before merge.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

[Copilot]

Copilot review overview

🔵 Needs a closer look

The new streaming state machine warrants human review, particularly its untested recovery path for actual stream errors.

Review effort: Balanced
Findings: 1

Open (1)

• Add regression test for partial output recovery after HTTP stream errors · New

What changed in this PR

Completes the Rust Responses API stack by adding streaming to the existing stateless endpoint.

Changes:

• Adds ordered lifecycle, reasoning, text, and function-call events with stable item IDs.
• Handles token limits, cancellation, and failures.
• Replaces streaming rejection coverage with six HTTP regressions.

File	Description
rust/​src/​server/​src/​routes/​tests/​responses.rs	Adds streaming lifecycle and output regressions.
rust/​src/​server/​src/​routes/​openai/​responses/​streaming.rs	Implements output-item event sequencing and ID tracking.
rust/​src/​server/​src/​routes/​openai/​responses.rs	Enables streaming transport and terminal response handling.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.