[MoE Refactor] Oracle Select FP8+NVFP4 Kernels In Priority#32414

Merged

mgoin merged 254 commits intomainfrom

Jan 21, 2026

This pull request is big! We're only showing the most recent 250 commits

Commits on Jan 8, 2026

stash
Robert Shaw
committed
first correctness!
Robert Shaw
committed
updated
Robert Shaw
committed
comments
Robert Shaw
committed
updated
Robert Shaw
committed
Merge branch 'main' into naive-dispatch-combine
robertgshaw2-redhat
authored
updateds
Robert Shaw
committed
nit changes
Robert Shaw
committed
support apply router weight on input
Robert Shaw
committed
attempt to get everything working for llama scout modelopt flashinfer cutlass
Robert Shaw
committed
updated
Robert Shaw
committed

Commits on Jan 10, 2026

apply to batched deep gemm
Robert Shaw
committed

Commits on Jan 11, 2026

updated
Robert Shaw
committed
stash
Robert Shaw
committed
remove NaiveBatchedExperts
Robert Shaw
committed
stash
Robert Shaw
committed
stash
Robert Shaw
committed
added back moe torch iterative
Robert Shaw
committed
revert changes
Robert Shaw
committed
re add
Robert Shaw
committed
add back iterative
Robert Shaw
committed
add methodology to kernels
Robert Shaw
committed

Commits on Jan 12, 2026

restructure kernel selection logic
Robert Shaw
committed
remove is_cuda
Robert Shaw
committed
added renamed file
Robert Shaw
committed
improve quant scheme
Robert Shaw
committed
improve validation
Robert Shaw
committed
improve platform selection logic
Robert Shaw
committed
nit newline
Robert Shaw
committed
nit newline
Robert Shaw
committed
revert spurious LOC change
Robert Shaw
committed
revert spurious LOC change
Robert Shaw
committed
updated
Robert Shaw
committed
update marlin
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
support differentiating on static vs dynamic
Robert Shaw
committed
newline nit
Robert Shaw
committed
update logic for aiter foudn
Robert Shaw
committed
cleanup support logic
Robert Shaw
committed
update logic for trtllm config
Robert Shaw
committed
updated
Robert Shaw
committed

Commits on Jan 13, 2026

more progress
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
Merge branch 'main' into naive-dispatch-combine
robertgshaw2-redhat
authored
merged
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
update kernel selection logic
Robert Shaw
committed
seem to have accuracy with dp/ep and tp for deepgemm
Robert Shaw
committed
updated
Robert Shaw
committed
re-add
Robert Shaw
committed
added back naive batched experts
Robert Shaw
committed
added back naive batched experts
Robert Shaw
committed
added back naive batched experts
Robert Shaw
committed
added back naive batched experts
Robert Shaw
committed
reduce LOC change
Robert Shaw
committed
reduce LOC change
Robert Shaw
committed
reduce LOC change
Robert Shaw
committed
reduce loc change
Robert Shaw
committed
reduce LOC change
Robert Shaw
committed

Commits on Jan 14, 2026

fix native ag/rs
Robert Shaw
committed
oracle is now working properly
Robert Shaw
committed
confirmed aiter env variables work as expected
Robert Shaw
committed
fix up oracle
Robert Shaw
committed
trying to make ct work
Robert Shaw
committed
Merge remote-tracking branch 'origin/main' into oracle-improvements
Robert Shaw
committed
made llama 4 work via compressed tensors
Robert Shaw
committed
initial attempt at modelopt
Robert Shaw
committed
attempt to fix naive multicast
Robert Shaw
committed
flashinfer appears to be working
Robert Shaw
committed
Merge remote-tracking branch 'origin/main' into oracle-improvements
Robert Shaw
committed
stash changes for review
Robert Shaw
committed
made modelopt start up
Robert Shaw
committed
still enable flashinfer
Robert Shaw
committed
remove comments
Robert Shaw
committed
appears that we have AG/RS working for nvfp4
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
updared comment
Robert Shaw
committed
make marling work with current device
Robert Shaw
committed
added compressed tensors nvfp4
Robert Shaw
committed
added compressed tensors nvfp4 - nit
Robert Shaw
committed
remove shared expert overlap functionality
Robert Shaw
committed
make cutedsl work
Robert Shaw
committed
hook up marlin experts properly
Robert Shaw
committed
hook up marlin experts properly
Robert Shaw
committed
make marlin work with AG/RS
Robert Shaw
committed
convert to using quant key
Robert Shaw
committed
convert to using quant key
Robert Shaw
committed
reject usage for things that have not migrated over yet
Robert Shaw
committed
reject usage for things that have not migrated over yet
Robert Shaw
committed
reject usage for things that have not migrated over yet
Robert Shaw
committed
reject usage for things that have not migrated over yet
Robert Shaw
committed
differentiate static vs dynamic quantization
Robert Shaw
committed
remove newline
Robert Shaw
committed
fix static vs dynamic
Robert Shaw
committed
get things working again
Robert Shaw
committed

Commits on Jan 15, 2026

updated
Robert Shaw
committed
updatred
Robert Shaw
committed
attempt to get sp working
Robert Shaw
committed
nit
Robert Shaw
committed
appears to be working properly
Robert Shaw
committed
fix pre commit
Robert Shaw
committed
updated
Robert Shaw
committed
remove flashinfer constructors
Robert Shaw
committed
nits
Robert Shaw
committed
remove do naive dispach combine comment
Robert Shaw
committed
update backend names
Robert Shaw
committed
update fallback experts
Robert Shaw
committed
fix up batched triton experts
Robert Shaw
committed
stash
Robert Shaw
committed
revert the device communicator
Robert Shaw
committed
updated
Robert Shaw
committed
Revert cutlass pf
Robert Shaw
committed
revert changes to layer
Robert Shaw
committed
revert chnge
Robert Shaw
committed
revert naive ep
Robert Shaw
committed
remove more unrelated changes
Robert Shaw
committed
cleanup flashinfer fp4 moe
Robert Shaw
committed
reduce LOC change
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
updated fp8
Robert Shaw
committed
updated 'make_fp8_moe_kernel'
Robert Shaw
committed
update fp8 moe layer
Robert Shaw
committed
updated the get_clss
Robert Shaw
committed
stash
Robert Shaw
committed
stahs
Robert Shaw
committed
Merge branch 'oracle-part-a' of https://github.com/vllm-project/vllm into oracle-part-a
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
make modelopt fp8 work
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
make nvfp4 marlin work
Robert Shaw
committed
updated
Robert Shaw
committed
updated logic for per_out_ch_qaunt
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
improve UX of kernel selection
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
merge mxfp4 hack
Robert Shaw
committed
remove deep gemm direct call
Robert Shaw
committed
nit
Robert Shaw
committed
fixing up tests
Robert Shaw
committed
fix deepgemm tests
Robert Shaw
committed
fix deepgemm tests
Robert Shaw
committed
trying to get deepgemm tests to pass
Robert Shaw
committed
stash
Robert Shaw
committed
fixing up kernel tests
Robert Shaw
committed
nice job by cursor bot on staticmethod decorator
Robert Shaw
committed
updating kernel tests so that they pass
Robert Shaw
committed
fix marlin
Robert Shaw
committed
remove test moe config
Robert Shaw
committed
updated
Robert Shaw
committed
various niceties for reducing loc change
Robert Shaw
committed
update test pipeline
Robert Shaw
committed
fix modular oai triton moe test
Robert Shaw
committed
udpate test
Robert Shaw
committed
reduce LOC change
Robert Shaw
committed
allow passing
Robert Shaw
committed
fix messed up config
Robert Shaw
committed
updated
Robert Shaw
committed
clean up unquanitzed
Robert Shaw
committed

Commits on Jan 16, 2026

fix re-commit
Robert Shaw
committed
unify marlin expers contruction
Robert Shaw
committed
updared
Robert Shaw
committed
Merge remote-tracking branch 'origin/main' into oracle-part-a
Robert Shaw
committed
pre commit
Robert Shaw
committed
fix nit on trtllmgenexperts'
Robert Shaw
committed
fix cruft aroundAITER
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed
updated is device capability family
Robert Shaw
committed
clean up rocm device support
Robert Shaw
committed
remove 'supports_mk_internally'
Robert Shaw
committed
clean up selection logic
Robert Shaw
committed
clean up fp8 oracle logic
Robert Shaw
committed
some nits
Robert Shaw
committed
clean up fp8 oracle
Robert Shaw
committed
fix routing method
Robert Shaw
committed
fix device capability family
Robert Shaw
committed
fix fp online quantization
Robert Shaw
committed
fix marlin fp8
Robert Shaw
committed
updated
Robert Shaw
committed
fix the llama router
Robert Shaw
committed
revert property/abstractmethod
Robert Shaw
committed
make mkt
Robert Shaw
committed
revert propery change
Robert Shaw
committed
address comment about on_gfx9
Robert Shaw
committed
Merge branch 'main' into oracle-part-a
robertgshaw2-redhat
authored
rearrange test group
Robert Shaw
committed
fix typing
Robert Shaw
committed
fix signature
Robert Shaw
committed
Merge branch 'main' into oracle-part-a
robertgshaw2-redhat
authored
fix typing
Robert Shaw
committed
nits
Robert Shaw
committed
fix FP8 DeepGEMM on Blackwell
Robert Shaw
committed
fix up configs
Robert Shaw
committed
fix quantization test, was falling back to marlin because we had wrong condition on triton experts
Robert Shaw
committed
Merge branch 'main' into oracle-part-a
robertgshaw2-redhat
authored
fix mk object creation
Robert Shaw
committed

Commits on Jan 17, 2026

Merge branch 'main' into oracle-part-a
robertgshaw2-redhat
authored
updated
Robert Shaw
committed
fix cutlass experts fp4
Robert Shaw
committed
improve ux
Robert Shaw
committed
updated
Robert Shaw
committed
address nits
Robert Shaw
committed

Commits on Jan 18, 2026

do the merge
Robert Shaw
committed
allow renormalzie and renormalize naive for fp8
Robert Shaw
committed
clean up routing logic
Robert Shaw
committed
remove routing method type from constrcutor
Robert Shaw
committed
fix some nits
Robert Shaw
committed
Merge branch 'main' into oracle-part-a
robertgshaw2-redhat
authored

Commits on Jan 19, 2026

merged
Robert Shaw
committed
updated
Robert Shaw
committed
updated
Robert Shaw
committed

Commits on Jan 20, 2026

address comment about get_clss()
Robert Shaw
committed
revert make batched vs make standard
Robert Shaw
committed
update mk objects
Robert Shaw
committed
update mk objects
Robert Shaw
committed
updated
Robert Shaw
committed
fix the signatures
Robert Shaw
committed
fix pre-commit
Robert Shaw
committed
Merge remote-tracking branch 'origin/main' into oracle-part-a
Robert Shaw
committed
remove llama 4.to_cuda
Robert Shaw
committed
Merge branch 'main' into oracle-part-a
robertgshaw2-redhat
authored
fix tess
Robert Shaw
committed
rename is gated act
Robert Shaw
committed
add unquantized to triton experts
Robert Shaw
committed
Merge branch 'oracle-part-a' of https://github.com/vllm-project/vllm into oracle-part-a
Robert Shaw
committed
remove batched deepgemm too
Robert Shaw
committed
fix typing
Robert Shaw
committed
fix nit
Robert Shaw
committed
backend_to_kernel_cls
Robert Shaw
committed
get_clses
Robert Shaw
committed
updated device typing
Robert Shaw
committed
Update vllm/model_executor/layers/fused_moe/oracle/nvfp4.py

robertgshaw2-redhat
and
pavanimajety
authored
Merge branch 'oracle-part-a' of https://github.com/vllm-project/vllm into oracle-part-a
Robert Shaw
committed
pre-commit
Robert Shaw
committed
raise error for not yet supported
Robert Shaw
committed

Commits on Jan 21, 2026

Merge branch 'main' into oracle-part-a
robertgshaw2-redhat
authored