Repository navigation
[Rust Frontend] Make unhandled vllm serve frontend flags explicit - #62431
Merged
BugenZhao merged 7 commits intoOct 20, 2026
Merged
Conversation
Python applies --custom-histogram-buckets to the frontend Prometheus histograms (vllm/v1/metrics/loggers.py). The Rust frontend did not declare it: vllm-rs serve forwarded it to the headless engine, and --args-json dropped it, so Rust kept its fixed buckets without a warning. Declare it as unsupported until the Rust histograms take configurable buckets. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
With --enable-tower-connector-lora, Python's InputProcessor prefixes the multimodal identifier with the LoRA name so encoder-cache entries do not collide across adapters. The Rust frontend did not declare the flag and always uses the bare UUID or hash, so the engine could reuse encoder outputs across adapters. Declare it as unsupported. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
With --enable-elastic-ep, Python's core client hands engine input sockets over during scaling and the API server exposes the scaling routes. The Rust frontend has neither and did not declare the flag, so it reached the engine while the frontend could not follow a scale event. Declare it as unsupported until elastic EP lands in Rust. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
The Rust frontend serves no OpenAPI, Swagger, or ReDoc endpoints, so it already behaves as if --disable-fastapi-docs were set. Rejecting the flag at startup only broke launch commands carried over from Python. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
…gine vllm-rs serve declared --structured-outputs-config as a no-op, so clap consumed it, printed a warning, and never passed it to the headless engine. The engine reads most of this config (disable_any_whitespace, disable_additional_properties, enable_in_reasoning, ...), so those options were lost. Stop declaring it so it is forwarded like other engine arguments. On the --args-json path the Python launcher already configures the engine itself, so the key is now ignored there instead of warned about. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
6 tasks done
…naged engine" This reverts commit 1d003fe. vllm-project#62456 parses --structured-outputs-config in the Rust frontend and forwards it to the managed engine itself, so this PR no longer touches the flag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
6 tasks done
Isotr0py
approved these changes
Oct 20, 2026
Member
Author
|
/ci run |
|
✅ Triggered Buildkite CI #96154 for commit ℹ️ This PR is 83 commits behind upstream |
6 tasks done
bantianyue
pushed a commit
to bantianyue/vllm
that referenced
this pull request
Oct 28, 2026
…llm-project#62431) Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: vllm-agent <claw@inferact.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Python
vllm servereads--custom-histogram-buckets,--enable-tower-connector-lora, and--enable-elastic-epin the frontend, but the Rust frontend did not declare them and dropped them without a warning. This PR makes them fail startup (Unsupported). It also accepts--disable-fastapi-docsas a no-op, since Rust serves no API docs routes.Claims
--custom-histogram-buckets,--enable-tower-connector-lora, and--enable-elastic-epnow fail startup on bothvllm-rs serveand the Python-supervised--args-jsonpath, instead of being silently ignored by the frontend.--disable-fastapi-docsis accepted with a no-op warning instead of failing startup. The Rust frontend serves no OpenAPI, Swagger, or ReDoc endpoints, so it already behaves as if the flag were set.Validation
Rust CI commands (
.buildkite/scripts/run-rust-frontend-cargo-ci.sh,style-clippyandtestmodes), run from the repository root at the branch tip:These ran before the structured-outputs commit was reverted; after the revert,
cargo nextest run -p vllm-cmd(82 passed) andcargo clippy -p vllm-cmd --all-targetsare clean.Each commit also passes workspace clippy with
-D warningson its own.New tests in
rust/src/cmd/src/cli/tests.rs:unsupported_args_fail_startup: each of the three flags failsvllm-rs serveparsing and is listed in the--args-jsonstartup error.noop_args_are_accepted:--disable-fastapi-docsparses on both paths.With
rust/src/cmd/src/cli/unsupported.rsreverted tomain, both new tests fail.Details
Where the code lives. Argument recognition belongs to the CLI layer. The flags reuse the existing
Unsupported/Noopmarkers inrust/src/cmd/src/cli/unsupported.rs, which already apply to both thevllm-rs serve(clap) and--args-json(serde) paths. No new mechanism is added.Why each flag is rejected rather than ignored.
--custom-histogram-bucketsreplaces the default buckets of the frontend Prometheus histograms (vllm/v1/metrics/loggers.py). The Rust histograms use fixed buckets, so the requested buckets were silently not applied.--enable-tower-connector-lora, Python'sInputProcessorprefixes the multimodal hash with the LoRA name so encoder-cache entries do not collide across adapters (vllm/v1/engine/input_processor.py). Rust always sends the bare hash, so with the flag on, the engine could reuse encoder outputs computed under a different adapter.--enable-elastic-ep, Python's core client hands engine input sockets over during scaling (vllm/v1/engine/core_client.py) and the API server exposes the scaling routes. The Rust frontend has neither, so it could not follow a scale event that the engine went through.--structured-outputs-config. An earlier revision forwarded it to the managed engine. #62456 parses it in the Rust frontend (it selects the structured-outputs backend) and forwards it itself, so this PR reverts that part and leaves the flag untouched.Behavior changes for existing users.
--disable-fastapi-docsnow starts instead of failing.Unsupportedflags were either not honored by the Rust frontend before or could produce wrong results with it.Not a duplicate. Searching open PRs for each flag finds no PR that changes how the Rust frontend handles them. #50698 (inactive since 2026-10-02) adds a supervisor-side control channel for elastic EP scaling but does not touch
--enable-elastic-ephandling in the Rust CLI. #50101 edits the neighboringstream_intervalentry inunsupported.rs, a different flag.AI assistance (Claude Code) was used to prepare this PR. The human submitter reviewed every changed line.
🤖 Generated with Claude Code
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.