[Bugfix] Fix DSV3 kernels breaking _C and _moe_C on unsupported arches#35123
Conversation
Signed-off-by: mgoin <mgoin64@gmail.com>
There was a problem hiding this comment.
Code Review
This pull request addresses build failures on unsupported architectures (e.g., SM121) by moving the ops.impl() registrations for DeepSeek V3 kernels from torch_bindings.cpp to their respective source .cu files. This change ensures that the implementation is only registered when the source file is actually compiled, removing hard symbol dependencies that previously caused link errors when CMake arch filtering excluded these files. The approach follows the established pattern in vLLM for architecture-specific kernels. I have identified one critical issue where a required header is missing in one of the modified source files, which would lead to compilation errors on supported platforms.
|
|
||
| #include "core/registration.h" |
There was a problem hiding this comment.
The addition of TORCH_LIBRARY_IMPL_EXPAND at the end of this file requires TORCH_LIBRARY_IMPL, which is defined in <torch/library.h>. While core/registration.h is included, it does not provide the underlying PyTorch macro definitions. Including <torch/all.h> (as seen in csrc/dsv3_fused_a_gemm.cu) or <torch/library.h> is necessary to avoid compilation errors on supported architectures.
#include <cuda_runtime.h>
#include <torch/all.h>
#include "core/registration.h"
|
loads fine fails on load (i think not related to this pr)
loads, fails on infer (again different story afaik) |
Cherry-picked from atalman/update_torch_211 (PR vllm-project#34644): - Bump torch version pins: 2.10.0 → 2.11.0 - Update CUDA version: 12.9 → 13.0 - CPU API changes: torch._C._cpu → torch.cpu - at::cpu::L2_cache_size() → get_cpu_capabilities() - Update test/build requirements for CUDA 13.0 - Fix distributed test for torch 2.11 - NIXL updates Rebased on synced main (includes upstream dsv3 fix vllm-project#35123).
vllm-project#35123) Signed-off-by: mgoin <mgoin64@gmail.com>
vllm-project#35123) Signed-off-by: mgoin <mgoin64@gmail.com>
vllm-project#35123) Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: Andrii Skliar <askliar@nvidia.com>
vllm-project#35123) Signed-off-by: mgoin <mgoin64@gmail.com>
vllm-project#35123) Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: EricccYang <yangyang4991@gmail.com>
Purpose
dsv3_fused_a_gemmanddsv3_router_gemmhad theirops.impl()intorch_bindings.cpp, creating a hard symbol dependency even when the.cufiles are excluded by CMake arch filtering. This causes the entire_C/_moe_Cextension to fail to link on architectures like SM121, taking down unrelated ops liketopk_softmax.See #34758 (comment) for reference of failure
Moved impl registration into the source
.cufiles viaTORCH_LIBRARY_IMPL_EXPAND, matching the existing pattern used by marlin, cutlass, and MLA kernels.Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.