Skip to content

[Rust Frontend] Make unhandled vllm serve frontend flags explicit - #62431

Merged
BugenZhao merged 7 commits into
vllm-project:mainfrom
BugenZhao:bz/rust-parity-serve-args
Oct 20, 2026
Merged

BugenZhao merged 7 commits into
vllm-project:mainfrom
BugenZhao:bz/rust-parity-serve-args

Conversation

@BugenZhao

@BugenZhao BugenZhao commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Overview

Python vllm serve reads --custom-histogram-buckets, --enable-tower-connector-lora, and --enable-elastic-ep in the frontend, but the Rust frontend did not declare them and dropped them without a warning. This PR makes them fail startup (Unsupported). It also accepts --disable-fastapi-docs as a no-op, since Rust serves no API docs routes.

Claims

  • --custom-histogram-buckets, --enable-tower-connector-lora, and --enable-elastic-ep now fail startup on both vllm-rs serve and the Python-supervised --args-json path, instead of being silently ignored by the frontend.
  • --disable-fastapi-docs is accepted with a no-op warning instead of failing startup. The Rust frontend serves no OpenAPI, Swagger, or ReDoc endpoints, so it already behaves as if the flag were set.

Validation

Rust CI commands (.buildkite/scripts/run-rust-frontend-cargo-ci.sh, style-clippy and test modes), run from the repository root at the branch tip:

cargo fmt --manifest-path rust/Cargo.toml --all -- --check   # ok
cargo sort --workspace --check rust                          # ok
cargo deny --manifest-path rust/Cargo.toml --config rust/deny.toml check bans   # ok
cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --all-features --locked -- -D warnings   # ok
cargo nextest run --manifest-path rust/Cargo.toml --workspace --all-features --locked --no-fail-fast
# 2142 tests run: 2142 passed, 13 skipped

These ran before the structured-outputs commit was reverted; after the revert, cargo nextest run -p vllm-cmd (82 passed) and cargo clippy -p vllm-cmd --all-targets are clean.

Each commit also passes workspace clippy with -D warnings on its own.

New tests in rust/src/cmd/src/cli/tests.rs:

  • unsupported_args_fail_startup: each of the three flags fails vllm-rs serve parsing and is listed in the --args-json startup error.
  • noop_args_are_accepted: --disable-fastapi-docs parses on both paths.

With rust/src/cmd/src/cli/unsupported.rs reverted to main, both new tests fail.

Details

Where the code lives. Argument recognition belongs to the CLI layer. The flags reuse the existing Unsupported / Noop markers in rust/src/cmd/src/cli/unsupported.rs, which already apply to both the vllm-rs serve (clap) and --args-json (serde) paths. No new mechanism is added.

Why each flag is rejected rather than ignored.

  • --custom-histogram-buckets replaces the default buckets of the frontend Prometheus histograms (vllm/v1/metrics/loggers.py). The Rust histograms use fixed buckets, so the requested buckets were silently not applied.
  • With --enable-tower-connector-lora, Python's InputProcessor prefixes the multimodal hash with the LoRA name so encoder-cache entries do not collide across adapters (vllm/v1/engine/input_processor.py). Rust always sends the bare hash, so with the flag on, the engine could reuse encoder outputs computed under a different adapter.
  • With --enable-elastic-ep, Python's core client hands engine input sockets over during scaling (vllm/v1/engine/core_client.py) and the API server exposes the scaling routes. The Rust frontend has neither, so it could not follow a scale event that the engine went through.

--structured-outputs-config. An earlier revision forwarded it to the managed engine. #62456 parses it in the Rust frontend (it selects the structured-outputs backend) and forwards it itself, so this PR reverts that part and leaves the flag untouched.

Behavior changes for existing users.

  • --disable-fastapi-docs now starts instead of failing.
  • The three new Unsupported flags were either not honored by the Rust frontend before or could produce wrong results with it.

Not a duplicate. Searching open PRs for each flag finds no PR that changes how the Rust frontend handles them. #50698 (inactive since 2026-10-02) adds a supervisor-side control channel for elastic EP scaling but does not touch --enable-elastic-ep handling in the Rust CLI. #50101 edits the neighboring stream_interval entry in unsupported.rs, a different flag.

AI assistance (Claude Code) was used to prepare this PR. The human submitter reviewed every changed line.

🤖 Generated with Claude Code


Pull Request Checklist
  • I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

  • AI assistance was used during the creation of this PR.

  • Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

  • Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

  • Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

  • Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

BugenZhao and others added 5 commits October 19, 2026 08:36
Python applies --custom-histogram-buckets to the frontend Prometheus
histograms (vllm/v1/metrics/loggers.py). The Rust frontend did not declare
it: vllm-rs serve forwarded it to the headless engine, and --args-json
dropped it, so Rust kept its fixed buckets without a warning. Declare it as
unsupported until the Rust histograms take configurable buckets.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
With --enable-tower-connector-lora, Python's InputProcessor prefixes the
multimodal identifier with the LoRA name so encoder-cache entries do not
collide across adapters. The Rust frontend did not declare the flag and
always uses the bare UUID or hash, so the engine could reuse encoder
outputs across adapters. Declare it as unsupported.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
With --enable-elastic-ep, Python's core client hands engine input sockets
over during scaling and the API server exposes the scaling routes. The Rust
frontend has neither and did not declare the flag, so it reached the engine
while the frontend could not follow a scale event. Declare it as
unsupported until elastic EP lands in Rust.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
The Rust frontend serves no OpenAPI, Swagger, or ReDoc endpoints, so it
already behaves as if --disable-fastapi-docs were set. Rejecting the flag
at startup only broke launch commands carried over from Python.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
…gine

vllm-rs serve declared --structured-outputs-config as a no-op, so clap
consumed it, printed a warning, and never passed it to the headless
engine. The engine reads most of this config (disable_any_whitespace,
disable_additional_properties, enable_in_reasoning, ...), so those options
were lost. Stop declaring it so it is forwarded like other engine
arguments. On the --args-json path the Python launcher already configures
the engine itself, so the key is now ignored there instead of warned about.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@BugenZhao
BugenZhao requested a review from njhill as a code owner October 19, 2026 08:44

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

…naged engine"

This reverts commit 1d003fe.

vllm-project#62456 parses --structured-outputs-config in the Rust
frontend and forwards it to the managed engine itself, so this PR no
longer touches the flag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@BugenZhao

Copy link
Copy Markdown
Member Author

/ci run

@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #96154 for commit f961ed6be6a7.

ℹ️ This PR is 83 commits behind upstream main. Merged upstream main into this branch as f961ed6be6a7 before starting CI. Pull the branch before pushing more changes.

@BugenZhao
BugenZhao merged commit 5528685 into vllm-project:main Oct 20, 2026
31 checks passed
bantianyue pushed a commit to bantianyue/vllm that referenced this pull request Oct 28, 2026
…llm-project#62431)

Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants