[errmakov]

Purpose

Adds OpenAI n > 1 support to the Rust frontend's /v1/chat/completions and /v1/completions, which today reject it with Only n=1 is supported.

This picks up #43780 by @Xunzhuo. A maintainer reviewed that PR ("The overall implementation LGTM"), but it then went stale with merge conflicts. Since then the routes have been reworked on main (response envelopes, ResponseOptions, continuous usage stats, cached/reasoning token details, SSE keep-alive), so this is a re-implementation of the same design on current main, not a rebase. @Xunzhuo is credited as co-author.

Like Python's ParentRequest (vllm/v1/engine/parallel_sampling.py):

• Each choice is submitted as its own child engine request. Child request IDs use the Python {index}_{parent} scheme when n > 1. The parent ID is kept unchanged for n = 1, so single-choice requests behave exactly as before.
• Each child's seed is offset by its choice index, so seeded choices differ from each other and stay reproducible.
• Usage counts the shared prompt once, including cached_tokens, and sums completion and reasoning tokens across choices.

Response shape:

• Streaming: the per-choice chunk streams get their index set and are interleaved (select_all). Each choice's final usage-only chunk is held back and merged, so clients get one usage chunk after all choices finish, followed by one [DONE]. continuous_usage_stats stays per choice, as in Python.
• Non-streaming: all choices are collected concurrently and merged into one response with indexed choices. Response-level fields such as prompt logprobs and kv_transfer_params come from the first choice.
• Errors: if any choice fails, the error propagates. The remaining child streams are dropped (and so aborted), the same way a disconnect is handled today.

Validation: /v1/completions now bounds n to 1..=10, the same range /v1/chat/completions already enforces, so one HTTP request cannot fan out without limit.

Not duplicating existing work: searched open and closed PRs for "rust n choices", "Rust Frontend n", "rust parallel sampling" and "Only n=1". The only PR for this change is #43780 itself. #52844 / #52399 are about /inference/v1/generate, which keeps rejecting n > 1.

AI assistance (Claude) was used to write this change. I reviewed every changed line and ran the tests below.

Test Plan

New route tests in rust/src/server/src/routes/tests.rs, using a mock engine that expects one request per choice:

• non_stream_{chat_completions,completions}_support_n_greater_than_one: checks the child request IDs (0_…, 1_…), the seed offsets (41 → 41/42), the indexed choices, and the merged usage.
• stream_{chat_completions,completions}_support_n_greater_than_one: checks per-index content, a single merged usage chunk as the last payload, and a single [DONE]. The chat variant also enables continuous_usage_stats and checks that per-choice usage survives the merge.
• completions_reject_n_outside_supported_range: n=0 and n=11 return 400.

Plus unit tests for the helpers in routes/openai/utils/parallel_sampling.rs.

cd rust
cargo test -p vllm-server
cargo clippy -p vllm-server --all-targets -- -D warnings
cargo fmt --check

Test Result

• cargo test -p vllm-server: 435 passed, 0 failed.
• All 5 new route tests fail against main's route code and pass with this change.
• cargo clippy -p vllm-server --all-targets -- -D warnings: clean.
• cargo fmt --check: clean.

No model eval was run. The change only affects the Rust frontend's request fan-out and response assembly, and for n = 1 the request it sends and the response it returns are unchanged.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @errmakov.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork