[WoosukKwon]

Overview

Follow-up to #61339 (vllm bench forward): remove its temporary dummy loader, now that #60096 has landed, and add ROCm TODOs.

Claims

• The bench's forward_bench_dummy loader is gone; --load-format dummy uses the stock DummyModelLoader. It ran an early process_weights_after_loading() so dummy-loaded DeepSeek-V4.1 could fuse its MegaMoE shared experts. [Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout #60096 loads MegaMoE experts directly into the DeepGEMM layout, so the CUDA models no longer have finalize_mega_moe_weights and the workaround was already a no-op.
• No timing change on DeepSeek-V4.1 (below).
• The model-microbenchmark skill drops its caveat about degenerate dummy norm weights, fixed by [Model Loader] Keep norm layers neutral under dummy weight init #61323.
• TODOs mark where ROCm support is missing (the bench is CUDA-only for now; its end-to-end test is skipped on ROCm because it hangs there, likely on the in-graph timing events).

Validation

pytest tests/benchmarks/test_forward_cli.py   # 3 passed (GB200)

DeepSeek-V4.1-Flash, DEP4 on one GB200 node, MegaMoE, FlashInfer sparse MLA, fp8 KV, ~100K KV, balanced routing; same environment and node, main's loader vs this PR:

case	main	this PR	diff
decode, 32 reqs	15565 us	15635 us	+0.45%
decode, 128 reqs	19986 us	20011 us	+0.12%
prefill 16K	205.1 ms	201.9 ms	-1.57%
prefill 8 x 1K	131.5 ms	129.7 ms	-1.32%

Diffs are within run-to-run noise (sample spread up to ~1% on V4.1 DEP). Graph-vs-eager and replay checks match between the two.

Details

• Unrelated observation, same on main: DeepSeek-V4.1 prefill graph replays are no longer bit-identical after main's DeepGEMM bump ([Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer #61149); decode replays still are. To be looked into separately.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

Not a duplicate: open PRs were searched for bench forward and the loader's name; none touch it.

AI assistance (Claude Code) was used to write this change and to run the tests and validation above.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[WoosukKwon]

/ci run

[github-actions]

✅ Triggered Buildkite CI #94895 for commit 20d988f7d52a.