Repository navigation
Conversation
|
/ci run |
|
❌ This PR is 2 commits behind upstream |
1832b54 to
580e85d
Compare
|
/ci run |
|
❌ This PR is 2 commits behind upstream |
The generic MXFP4 selector lists the candidate backends and each one's rejection reason when nothing matches. The DeepSeek-V4 selector logged the same reasons at debug level and then raised a bare "No MXFP4 MoE backend supports the deployment configuration", so a server that refuses to start gives no way to tell a missing kernel from, for example, the batch invariance gate in modular_kernel.is_supported_config -- without reading the source. Collect the reasons and put them in the error, matching the wording the generic path already uses. Signed-off-by: Ao Shen <aoshen@inferact.ai> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
|
/ci run |
580e85d to
a482ac2
Compare
|
❌ This PR is 2 commits behind upstream |
Summary
Trying to start DeepSeek-V4.1-Flash with
VLLM_BATCH_INVARIANT=1on a GB200,every worker died with exactly this and nothing else:
The generic MXFP4 selector in the same file raises with the diagnosis attached:
select_deepseek_v4_mxfp4_moe_backendcollects the same per-backend reasons,logs them at
debug_once, and then throws them away. So on the models mostlikely to land in an unusual configuration, the failure says the least.
In my case the reason was the batch-invariance gate in
modular_kernel.is_supported_config(envs.VLLM_BATCH_INVARIANT and not cls._supports_batch_invariance()), which I could only find by reading thesource. With this PR the error says it.
Change
Collect the rejection reasons in the DeepSeek-V4 path and include them in the
error, with the wording the generic path already uses.
Not a duplicate
gh pr list --search "mxfp4 backend selection error"returns backend additionsand unrelated fixes (#55232, #54032, #55275, #54109); none of them touches the
error path.
Test plan
CPU-only. The new test stubs a kernel class that refuses with a known reason and
asserts both the candidate list and that reason reach the exception.
No behavior change beyond the message: the same configurations succeed and the
same ones fail.
AI assistance
Developed with AI assistance (Claude). This came out of an actual multi-node run
on DeepSeek-V4.1-Flash, not a code read — the empty message is what cost the
time. I reviewed every changed line, ran the tests above, and I am the
accountable submitter.
🤖 Generated with Claude Code
Review rounds
An adversarial automated review was run over this diff against
origin/main,asked specifically whether the collection loop changes which backend is
selected, whether the new error path is reachable where the old code succeeded
(or vice versa), whether each reported reason is the actual rejection cause,
whether any backend is now probed that was previously skipped (import side
effects, CUDA init), and whether the report can leak an internal name or come
out empty or duplicated. It found no defects.