Repository navigation
[Bugfix][Rust Frontend] Rust vllm-bench accepts --max-concurrency 0 and hangs forever (Python rejects it since #54887) - #64390
Conversation
Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com>
|
Fix looks correct. Two requests before merge: |
…natory comments Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com>
|
Addressed both requests in commit edc704d: (1) (2) Trimmed the explanatory comments in Verification: |
|
/ci run |
|
✅ Triggered Buildkite CI #98247 for commit ℹ️ This PR is 1239 commits behind upstream |
|
✅ @SIDDARTHAREDDY8, CI is now available for this PR.
|
Fixes #64319.
Rust
vllm-benchaccepted--max-concurrency 0and then hung forever onSemaphore::new(0), while Pythonvllm bench serverejects 0 since #54887. This change rejects 0 fail-fast on every Rust bench path that builds a concurrency semaphore:config.rs):--max-concurrency 0and--multi-turn-concurrency 0are rejected with a clear error instead of hanging.sweep.rs): any 0 in--sweep-max-concurrencyis rejected before the first sweep point runs.mm-processorpath (mm_processor.rs):--max-concurrency 0is rejected at the entry point, before the managed engine is spawned, since this subcommand does not go throughBenchConfig::from_args.Why this is not duplicating an existing PR: a search for open PRs mentioning #64319 returns only this PR, and #64319 itself has no linked PR.
Tests run:
cargo test -p vllm-bench --lib: 191 passed, 0 failed, 12 ignored.cargo test -p vllm-bench --lib concurrency: 8 passed, 0 failed (the zero/positive max-concurrency, multi-turn, and sweep cases).rustfmtclean under the workspacerustfmt.toml(style edition 2024, chain width 80).No output, accuracy, or serving behavior changes, so no model evaluation results apply. No new CLI args or defaults, so no documentation update is needed.
AI-assistance disclosure: AI assistance was used in preparing this change.