Skip to content

Support MLOPart imports from vendored package names - #32

Merged
zyongye merged 3 commits into
vllm-project:devfrom
cleonard530:add_relative_deep_gemm_import
Oct 13, 2026
Merged

zyongye merged 3 commits into
vllm-project:devfrom
cleonard530:add_relative_deep_gemm_import

Conversation

@cleonard530

@cleonard530 cleonard530 commented Oct 12, 2026 •

Copy link
Copy Markdown

Description

DeepGEMM’s native MLOPart bridge hardcoded deep_gemm.utils.mlopart, which fails when DeepGEMM is vendored under another package name such as vllm.third_party.deep_gemm.
This change:

  • Passes the active Python package name during initialization.
  • Resolves utils.mlopart relative to that package.
  • Keeps torch.ops.deep_gemm.init(path) backward compatible by defaulting the package name to deep_gemm.

Testing

  • Forced extension rebuild completed successfully.
  • Verified standard deep_gemm import.
  • Verified legacy one-argument torch.ops.deep_gemm.init(path).
  • Verified native MLOPart lookup resolves through a synthetic vendored package.

I think this will need to go through before vllm-project/vllm#61149 can be merged into vllm.

Co-authored-by: codex
reviewed by @cleonard530

cc @Harry-Chen @janeyx99

Co-authored-by: codex

Signed-off-by: Chris Leonard <chleonar@redhat.com>
Signed-off-by: Chris Leonard <chleonar@redhat.com>
@cleonard530
cleonard530 marked this pull request as ready for review October 12, 2026 17:30
@zyongye

zyongye commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Hi @cleonard530 could you check if #31 has solve the issue? Thanks

Edit:
ok I guess not. I will let @Harry-Chen and @janeyx99 to review

@zyongye

zyongye commented Oct 12, 2026

Copy link
Copy Markdown
Member

While checking whether anything else in the tree made the same package-name assumption, I found one more that would fit naturally in this PR.

deep_gemm/_C.py builds the locality_domain shim as f'{__name__}.locality_domain' so the name follows the package, but the epilogue shim a few lines above still hardcodes it:

-epilogue = types.ModuleType('deep_gemm.epilogue', 'GEMM epilogue classes')
+epilogue = types.ModuleType(f'{__package__}.epilogue', 'GEMM epilogue classes')

Nothing breaks today: epilogue is reached as an attribute and is never registered in sys.modules, so the stale name only shows up in repr()/help(). It would turn into a real phantom deep_gemm.epilogue entry for anyone who later mirrors the sys.modules registration that locality_domain has. __package__ keeps the name identical for a normal install and tracks the vendored package otherwise — verified on top of this branch:

import before after
import deep_gemm deep_gemm.epilogue deep_gemm.epilogue
vendored under another package deep_gemm.epilogue <vendored>.epilogue

I would have pushed it to this branch directly — maintainer_can_modify is enabled — but your fork's parent is deepseek-ai/DeepGEMM, so admin on the sibling vllm-project/DeepGEMM doesn't grant maintainer-edit access there (403). Feel free to take the one-liner, or I can send it as a follow-up PR if you'd rather keep this one focused.

For what it's worth, the rest of the audit came back clean, and this PR does fix the real thing:

  • mlopart.hpp:88 was the only hardcoded Python import in csrc/; after this PR there are none.
  • The MLOPart server spawn already used [sys.executable, '-I', __file__, ...] rather than -m, so it was vendoring-safe.
  • Every internal import in the package is relative.
  • Reproduced the failure on current dev by importing the package under another name: RuntimeError: Assertion error (csrc/runtime/mlopart.hpp:62): MLOPart Python call failed: No module named 'deep_gemm'. With this branch built, the same test returns is_localization_available() -> True, and both import deep_gemm and the legacy one-arg torch.ops.deep_gemm.init(path) still work.

@cleonard530

Copy link
Copy Markdown
Author

Feel free to take the one-liner, or I can send it as a follow-up PR if you'd rather keep this one focused.

I'll update this real quick

Signed-off-by: Chris Leonard <chleonar@redhat.com>
@zyongye
zyongye merged commit 6ce1c5e into vllm-project:dev Oct 13, 2026
1 check passed
zyongye added a commit to zyongye/vllm that referenced this pull request Oct 13, 2026
…parse indexer

Pin vllm-project/DeepGEMM dev 6ce1c5e: the upstream 26/09/30 release
(vllm-project/DeepGEMM#30), the vendoring import fixes
vllm-project/DeepGEMM#31 and vllm-project/DeepGEMM#32, and the paged
sparse MQA page-stride relaxation vllm-project/DeepGEMM#33. Vendor the
new deep_gemm/locality_domain.py. DeepGEMM#32 makes the MLOPart bridge
and the epilogue module use the package's own name, so they also work
when vendored as vllm.third_party.deep_gemm. DeepGEMM#33 lowers the
page-stride guard from 512 to 256 bytes; the DeepSeek-V4.1
sparse-logits block stride stays 512-aligned, which still satisfies it.

The MQA logits API changed: clean_logits/logits_dtype are gone,
max_seqlen_k is required, prefill logits are compressed to each row's
K span, and SM100 takes only MXFP8/MXFP4 with BF16 weights and returns
BF16 logits. On SM100 the FP8 indexer K writers store their existing
power-of-two scale as four packed UE8M0 bytes (a lossless MXFP8
encoding), FP8 Q passes unit MX scales with its scale still folded into
the weights, and BF16 logits are upcast row-aware for the FP32 top-k
kernels. check_wheel_deepgemm.py now imports the vendored package.

build_deepgemm_C.py passes --force to setup.py build_ext: DeepGEMM's
setup.py only tracks python_api.cpp, so an incremental build after a
header-only pin change kept a stale extension. CMake already decides
when this step reruns.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants