Repository navigation
Support MLOPart imports from vendored package names - #32
Conversation
Co-authored-by: codex Signed-off-by: Chris Leonard <chleonar@redhat.com>
Signed-off-by: Chris Leonard <chleonar@redhat.com>
|
Hi @cleonard530 could you check if #31 has solve the issue? Thanks Edit: |
|
While checking whether anything else in the tree made the same package-name assumption, I found one more that would fit naturally in this PR.
-epilogue = types.ModuleType('deep_gemm.epilogue', 'GEMM epilogue classes')
+epilogue = types.ModuleType(f'{__package__}.epilogue', 'GEMM epilogue classes')Nothing breaks today:
I would have pushed it to this branch directly — For what it's worth, the rest of the audit came back clean, and this PR does fix the real thing:
|
I'll update this real quick |
Signed-off-by: Chris Leonard <chleonar@redhat.com>
…parse indexer Pin vllm-project/DeepGEMM dev 6ce1c5e: the upstream 26/09/30 release (vllm-project/DeepGEMM#30), the vendoring import fixes vllm-project/DeepGEMM#31 and vllm-project/DeepGEMM#32, and the paged sparse MQA page-stride relaxation vllm-project/DeepGEMM#33. Vendor the new deep_gemm/locality_domain.py. DeepGEMM#32 makes the MLOPart bridge and the epilogue module use the package's own name, so they also work when vendored as vllm.third_party.deep_gemm. DeepGEMM#33 lowers the page-stride guard from 512 to 256 bytes; the DeepSeek-V4.1 sparse-logits block stride stays 512-aligned, which still satisfies it. The MQA logits API changed: clean_logits/logits_dtype are gone, max_seqlen_k is required, prefill logits are compressed to each row's K span, and SM100 takes only MXFP8/MXFP4 with BF16 weights and returns BF16 logits. On SM100 the FP8 indexer K writers store their existing power-of-two scale as four packed UE8M0 bytes (a lossless MXFP8 encoding), FP8 Q passes unit MX scales with its scale still folded into the weights, and BF16 logits are upcast row-aware for the FP32 top-k kernels. check_wheel_deepgemm.py now imports the vendored package. build_deepgemm_C.py passes --force to setup.py build_ext: DeepGEMM's setup.py only tracks python_api.cpp, so an incremental build after a header-only pin change kept a stale extension. CMake already decides when this step reruns. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Description
DeepGEMM’s native MLOPart bridge hardcoded deep_gemm.utils.mlopart, which fails when DeepGEMM is vendored under another package name such as vllm.third_party.deep_gemm.
This change:
Testing
I think this will need to go through before vllm-project/vllm#61149 can be merged into vllm.
Co-authored-by: codex
reviewed by @cleonard530
cc @Harry-Chen @janeyx99