[feat] Add routing_replay_out support to MoE kernels and Python API#3024

Merged

aleozlx merged 24 commits intoflashinfer-ai:mainfrom

TomerBN-Nvidia:upstream-routing-replay

Apr 14, 2026

Commits on Apr 12, 2026

feat: add routing_replay_out to MoE kernel launchers and routing kernels
Tomer Natan
committed
feat: add routing_replay_out to noAuxTc DSV3 routing kernel
Tomer Natan
committed
feat: add routing_replay_out to Python MoE API
Tomer Natan
committed
test: add routing replay tests for FP8 and DSV3 MoE kernels
Tomer Natan
committed
docs: add vLLM routing replay integration guide
Tomer Natan
committed
fix: add missing int16 dtype validation and FP4/MXINT4 public API params
Tomer Natan
committed
fix: address code review feedback — missing routing_replay_out params, duplicate test, Llama4 replay gaps

TomerBN-Nvidia
authored and
Tomer Natan
committed
fix: clarify unordered column semantics in doc and strengthen replay test

TomerBN-Nvidia
authored and
Tomer Natan
committed
docs: clarify API list is vLLM-specific subset, add language tag to code block

TomerBN-Nvidia
authored and
Tomer Natan
committed
fix: add routing_replay_out to FP8 per-tensor op/fake_op, test oversized buffer contract

TomerBN-Nvidia
authored and
Tomer Natan
committed
fix: add __version__ to flashinfer_cubin for nightly base compat
Tomer Natan
committed
fix: wrap DLDataType brace initializer in extra parens for TVM_FFI_ICHECK
Tomer Natan
committed

Commits on Apr 13, 2026

fix: address human review — revert __version__, docstring, trtllm-only doc, validation helper, DeepSeek-only replay

TomerBN-Nvidia
authored and
Tomer Natan
committed
fix: add missing Optional import to tvm_ffi_utils.h for noAuxTcKernels JIT compilation
Tomer Natan
committed
fix: switch FP8 block scale replay test from Renormalize to DeepSeekV3 routing
Tomer Natan
committed
fix: column order matches topk_indices, remove em dash
Tomer Natan
committed
fix: reject strided routing_replay_out views - require contiguous layout
Tomer Natan
committed
fix: move Optional import to noAuxTcKernels, add Python validation, swap condition order

TomerBN-Nvidia
authored and
Tomer Natan
committed
Merge branch 'main' into upstream-routing-replay
aleozlx
authored