Repository navigation
[Fix] moe: Publish a server config in the CUTLASS runner dispatch-format and placement tests - #45376
Merged
ShangmingCai merged 1 commit intoOct 20, 2026
Conversation
This comment has been minimized.
This comment has been minimized.
Collaborator
|
/tag-and-rerun-ci |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #43363, merged today: it broke two CPU unit tests on
main, and this test-only PR fixes them. Sorry for the breakage.cc @Fridge003 (merged #43363; added the dispatcher test in #44666) and @Jiminator (added the placement test in #44518).
Ready to run:
/rerun-test test/registered/unit/layers/moe/test_flashinfer_dispatcher.py.test_moe_runner_config_placement.pyruns only in the Intel CPU suite (stage-b-test-cpu-intel), so/rerun-teston ubuntu-latest skips it.Summary
mainsince #43363 merged. Both call the FlashInfer CUTLASS MoE runner directly without publishing server args. #43363 made the runner read thedisaggconfig to size its tuning range, so they now raiseValueError: config namespace 'disagg' not published.get_context().override_server_args(model_path="dummy"), the same pattern the neighbouring tests in these files already use.Motivation
Since #43363, the CUTLASS MoE runner (
python/sglang/srt/layers/moe/moe_runner/flashinfer_cutlass.py) readsget_disagg()and the prefill-chunk settings on every call. That PR also updated the tests that existed at its base. Two tests were added onmainafter that base, and they call the runner without any server args published:mainf8fd41c36e4test/registered/unit/layers/moe/test_flashinfer_dispatcher.py::TestFlashinferRunnerDispatchFormats::test_cutlass_preserves_prefill_and_decode_combine_formatsbase-a-test-cputest/registered/unit/layers/moe/test_moe_runner_config_placement.py::test_sm120_nvfp4_dispatch_uses_live_layer_placementstage-b-test-cpu-intelBoth fail with
ValueError: config namespace 'disagg' not published, raised from_cutlass_moe_tune_max_num_tokens. A real server always publishes these configs, so serving is not affected. Only these tests are.Modifications
test_flashinfer_dispatcher.py: addget_context().override_server_args(model_path="dummy")to the existingwith (...)block aroundfused_experts_flashinfer_to_flashinfer_cutlass.test_moe_runner_config_placement.py: wrap the_run_flashinfer_cutlasscall in the same context manager. This is the pattern_kernel_placementin the same file uses.These two tests and #43363's own
test_flashinfer_cutlass_swiglu_params.pyare the only tests undertest/that call the CUTLASS MoE runner entry points directly.Accuracy Tests
Not applicable: no runtime code changes.
Speed Tests and Profiling
Not applicable: no runtime code changes.
Tests
Run CPU-only, the way CI runs them (
cd test && python3 registered/<file>):mainf8fd41c36e4unit/layers/moe/test_flashinfer_dispatcher.pyunit/layers/moe/test_moe_runner_config_placement.pyunit/layers/moe/test_flashinfer_cutlass_swiglu_params.py(unchanged)pre-commit runon the two changed files passes.Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ciCI States
Latest PR Test (Base): ⏳ Run #39407165818
Latest PR Test (Extra): ❌ Run #39407165706
Latest PR Test (AMD ROCm 10): ⏳ Run #39407165709