WIP: b12x updates - #41243

Closed

askliar wants to merge 54 commits into

vllm-project:mainfrom

askliar:askliar/b12x-with-tinygemm

Commits on Apr 20, 2026

[SM120] Add FlashInfer CuteDSL SM12x fused MoE backend

meena-at-work
authored and
Andrii Skliar
committed
16fa2c5
[SM12x] Fix test_cutedsl_sm12x_moe numerical mismatch and add expects_unquantized_inputs

authored and
Andrii Skliar
committed
3385937
SM12x: fix production path + kernel.py Literal + test GPU name

authored and
Andrii Skliar
committed
9705fe5
Fix E501 line-too-long in SM12x CuteDSL files

Andrii Skliar
and
claude
committed
e586b90
small fix
Andrii Skliar
committed
6e11cd7
undo unnecessary changes
Andrii Skliar
committed
b78f654
Refactor and rename FlashInfer CuteDSL SM12x to B12x
Andrii Skliar
committed
5bccd29
Fix B12x MoE on SM121 (DGX Spark): MMA layout conversion and capability family

Andrii Skliar
and
claude
committed
6cbf034
Use FlashInfer tinygemm_bf16 for small-M BF16 GEMVs (decode optimization)

Andrii Skliar
and
claude
committed
f5282b4
Fix tinygemm dispatch for multi-dimensional input tensors

Andrii Skliar
and
claude
committed
d86cf09
Fix torch.compile compatibility for tinygemm dispatch

Andrii Skliar
and
claude
committed
38508ee
Fix torch.compile (AOT fullgraph) compatibility for tinygemm

Andrii Skliar
and
claude
committed
9e431b3
Refactor tinygemm to use vLLM custom op pattern for torch.compile

Andrii Skliar
and
claude
committed
dca202e
fix local_expert_offset arg
Andrii Skliar
committed
2d4ded1
Add tests for FlashInfer tinygemm BF16 functionality
Andrii Skliar
committed
ee67a74
add torchinductor trick
Andrii Skliar
committed
3389426
add more improvements from b12x
Andrii Skliar
committed
575bb9d
add tests
Andrii Skliar
committed
21bc427
[SM120/121] Add FlashInfer b12x FP4 GEMM backend

authored and
Andrii Skliar
committed
11f2330
extend is_big_gpu override with Blackwell arch + inductor config

Andrii Skliar
and
claude
committed
4b4a1bc
Refactor BF16 linear dispatch logic and enhance tests
Andrii Skliar
committed
7d81fce

Commits on Apr 21, 2026

Enhance inductor autotune configuration to include CUTLASS backend
Andrii Skliar
committed
48a86ba
Set inductor cutlass_dir fallback for container/pip environments
root
committed
d43e28a
Use static-shape compiled linears (dynamic=False, fullgraph=True)
root
committed
f18267b
Refactor inductor configuration and enhance BF16 linear dispatch tests
Andrii Skliar
committed
b2f9ac4

Commits on Apr 22, 2026

add specdec synthetic; fix torch-compile
Andrii
committed
c92afb1
Merge remote changes, resolve utils.py conflict (take ours)
Andrii
committed
3d4b81a

Commits on Apr 24, 2026

Refactor inductor configuration and enhance BF16 linear dispatch tests
Andrii Skliar
committed
98f2214
Merge remote-tracking branch 'upstream/main' into askliar/b12x-with-tinygemm
Andrii Skliar
committed
4d022d2

Commits on Apr 28, 2026

Refactor inductor configuration and enhance BF16 linear dispatch tests
Andrii Skliar
committed
c7fd0d7
Add routing tests for FlashInfer spec-decode path
Andrii Skliar
committed
bec4c85
Enhance FlashInfer tests and metadata handling
Andrii Skliar
committed
4757c79
trim down the PR
Andrii Skliar
committed
0c3589e
minor style fixes
Andrii Skliar
committed
3cb0807
pre-commit changes
Andrii Skliar
committed
e681034
trim the PR furtherg
Andrii Skliar
committed
66a0a4c

Commits on May 4, 2026

Merge remote-tracking branch 'upstream/main' into askliar/b12x-with-tinygemm
Andrii Skliar
committed
3cc189b
Merge remote-tracking branch 'origin/feat/add_fi_full_cudagraph_with_mtp' into askliar/b12x-with-tinygemm
Andrii Skliar
committed
cd5ef31

Commits on May 9, 2026

add proper support for cutlass for prefill and cutlass for decode
Andrii
committed
88300db

Commits on May 12, 2026

:Merge branch 'main' of https://github.com/vllm-project/vllm into askliar/b12x-with-tinygemm
Andrii
committed
ddc39a3
add nemotron_3_5_fixes
Andrii
committed
ab7807d

Commits on May 16, 2026

fix b12x integration
Andrii
committed
d6304d3

Commits on May 17, 2026

b12x: fix W4A16 intermediate-size padding sync + accept renamed gemm class

Andrii
and
claude
committed
b9b35ab
Fix "all" mode support with SpecDec

roikoren755
authored and
Andrii
committed
33f9da4
Delete some tests

roikoren755
authored and
Andrii
committed
946fd90
Revert "Default to 'align' mamba cache mode for Mamba-based models when speculative decoding is enabled (#40454)"

roikoren755
authored and
Andrii
committed
1380664
Fix mypy

roikoren755
authored and
Andrii
committed
5616145
CR

roikoren755
authored and
Andrii
committed
7fc00fd
Plumb prev-step input anchor for all-mode + SpecDec

roikoren755
authored and
Andrii
committed
9d8faf2
Fix state indices

roikoren755
authored and
Andrii
committed
fa838b9
Small refactor

roikoren755
authored and
Andrii
committed
cb406b0
Inline function and delete tests

roikoren755
authored and
Andrii
committed
fc77b2e
Final cleanup

roikoren755
authored and
Andrii
committed
6b44356
Skip blocking GPU->CPU sync of num_accepted_tokens in hybrid+align mode when no mamba block boundary can be crossed

committed
4f034dc