[WoosukKwon]

Overview

Day-0 support for Reflection's Beam-1.0 (BeamForCausalLM), a 501B-parameter MoE with 23B active parameters, and its DFlash2 drafter (DFlash2BeamDraftModel).

This PR is now a tracking PR and is not meant to be merged. It was split into the reviewable PRs below; everything on this branch is either in main or in an open slice.

Status

PR	Scope	Status
#62042	Router GEMM kernels for (5120, 256) and BF16 output in ll_bf16_gemm	✅ Merged (2026-10-16)
#62052	beam reasoning and tool parsers, VLLM_BEAM_ALWAYS_CONSTRAIN_OUTPUT	✅ Merged (2026-10-16)
#62074	Beam model and DFlash2 drafter (Blackwell kernels, standard TP all-reduce)	✅ Merged (2026-10-19)
#62366	Fused MNNVL all-reduce with Beam's sandwich norms and MoE finalize	🔄 In review
#62073	Attention DP + CuTeDSL NVFP4 MegaMoE (needs FlashInfer nightly); stacked on #62366	🔄 In review

What landed differs from this branch:

• BeamConfig and the model follow the released config.json schema (layer_types, mlp_layer_types, per-type rope_parameters). Handling only pre-release checkpoints needed was removed, and switches the release never sets are fixed. Code built on this branch that reads the removed keys (for example enable_sandwich_norm, mlp_only_layers, nope_layer_ids, moe_router_dtype) must use the released behavior instead.
• The DFlash2 drafter no longer subclasses the Qwen3 DFlash classes.
• The DFlash max_position_embeddings override was dropped: the released drafter already matches the target.

Remaining work:

• Serving re-run on main (NVFP4 + DFlash2, TP8 GB200) with the released checkpoints. The numbers below were measured on this branch with the full stack, including the fused all-reduce.
• [Model][Perf] Fuse Beam's TP all-reduce with its sandwich norms and MoE finalize #62366: end-to-end before/after against main, and whether allreduce.cu (and [Beam] Enable CuTeDSL NVFP4 MegaMoE #62073's moe_finalize.cu) stay runtime-JIT through FlashInfer or are built into vLLM.
• [Beam] Enable CuTeDSL NVFP4 MegaMoE #62073: validation with FlashInfer nightly, which the MegaMoE paths and the DP-boundary test require.

Validation (this branch, full stack)

Serving: Beam-1.0 NVFP4 + DFlash2 (7 draft tokens), TP8 on GB200, SPEED-Bench throughput_1k low-entropy subset, greedy decoding:

Concurrency	Fixed draft (tok/s)	Adaptive (tok/s)	Acceptance length
1	708	707	3.71 → 3.61
128	15,012	16,632	3.68 → 3.23

Prefill, 4×GB300, 32K input / 1 output token:

Configuration	Peak input throughput (tok/s)
Attention DP + MegaMoE	168.8K
TP4	98.7K

Decode/prefill kernels: compared with the first implementation, TP8 batch-1 decode went from 8.95 ms to 3.30 ms per step (2.71×), and an 8 × 2K prefill from 328 ms to 176 ms (1.87×). Part of this comes from the fused all-reduce in #62366.

vllm serve reflection/Beam-1.0-501B-A23B-NVFP4 -tp 4 --max-num-seqs 128 \
  --reasoning-parser beam --tool-call-parser beam --enable-auto-tool-choice \
  --kv-cache-dtype fp8 --attention-config '{"backend":"FLASHINFER"}' \
  --kernel-config '{"enable_flashinfer_autotune":false}' \
  --speculative-config '{"method":"dflash","model":"reflection/Beam-1.0-501B-A23B-DFlash2","num_speculative_tokens":7,"attention_backend":"FLASH_ATTN"}'

AI assistance (Claude Code) was used; the submitter reviewed every changed line.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[mergify]

Documentation preview: https://vllm--61978.org.readthedocs.build/en/61978/

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @WoosukKwon.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork