Repository navigation
refactor(musa): bind DeepSeek-V4 optimizations to model contract - #167
Merged
yeahdongcn merged 8 commits intoAug 4, 2026
Merged
Conversation
added 8 commits
August 3, 2026 19:30
Collaborator
Author
|
Consolidated into #166. The Qwen branch was fast-forwarded from |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR extends the optimization-contract infrastructure from #166 to
DeepSeek-V4. It is intentionally stacked on
feat/musa-60083-qwen-contract; reviewers should review only the eight commitsafter
e2c012dde.The contract combines immutable model facts from
config.jsonwith runtimeexecution facts (parallel topology, KV dtype, attention backend, compilation
mode, CUDA-graph mode, and max sequences). Consumers bind the resolved
contract to their owning module and fail closed when identity or execution
metadata is incomplete.
The DeepSeek-V4 contract now owns selection for:
Unsupported model identities, shapes, quantization layouts, parallel
topologies, or execution modes retain the existing path. Explicit generic
kernel overrides remain available and keep precedence over contract-selected
defaults.
Why
DeepSeek-V4 optimizations previously mixed model-name checks, shape checks,
process-wide environment mutation, and consumer-local assumptions. That made
selection difficult to audit and could leak one model's tuning into another
model loaded by the same process.
This change centralizes the static and runtime contract while leaving the
optimized kernels and vLLM-Omni integration boundary unchanged.
Performance alignment with #156
This stack is based on #156, so it does not claim the original +4.46%
shared-MLP fusion gain again. The performance gate here is preservation of the
#156 result while replacing its runtime selection plumbing.
The exact #156 TP8 workload was repeated on one isolated 8 x S5000 node with
the 5.2.0-server driver and image
registry.mthreads.com/mcconline/inference/vllm:v0.24.0-ph1-5.2.0-torch2.9.1.post1-20260725:/home/dist/models/DeepSeek-V4-Flash-Base, BF16 model, block-FP8 weights;NONE,FULL_DECODE_ONLY, capture sizes[1], copy inputs off;e2c012dde, includes #156)4d0572f12)All four measurements completed 1/1 requests, generated 1024 tokens, and had
zero failures.
For direct comparison, #156 reported a controlled candidate mean of
51.0648 tok/s, 779.92 ms TTFT, and 18.7415 ms TPOT. This candidate differs by
-0.40% TPS, +1.26% TTFT, and +0.37% TPOT, within a 3% repeatability envelope.
The controlled current-stack A/B is effectively neutral, so the contract
migration does not remove the #156 optimization.
Validation
Candidate source:
4d0572f12c450c24a9bbfc3c7e2b4fae306a9cb9.Pinned upstream vLLM:
ee0da84ab9e04ac7610e28580af62c365e898389.171 passed, 2 skipped;112 passed;64 passed;17 passed;6 passed;EngineArgsforDeepSeek-V4-Flash-Baseresolved thedeepseek_v4.tp8_flash_baseprofile;90c592a2dc666e9352a9bc28387414486f074351c7f57a1028bd528e0d0243bc;the new
0101-MUSA-bind-DeepSeek-V4-optimization-contract.patch;ruffpasses for the new contract modules and focused DSV4 tests, andgit diff --checkpasses for the complete stack. The existing import-orderfindings in
tests/test_patches.pyand the lazy DeepGEMM import block areunchanged from the stack base.
Model regression on the final candidate SHA
VLLM_COMPILE,FULL_DECODE_ONLY, FLASH_ATTNBeijing, health 200quantization=fp8,Beijing, health 200Qwen3MoeForCausalLM,Beijing, health 200Beijing, final health 200Beijing, health 200The BF16 MoE/MLA fallback smoke currently stops before serving on
UnquantizedFusedMoEMethod.is_monolithic. The same exception reproduces withthe
e2c012ddecontrol image, so it is recorded as a pre-existing stackbaseline failure rather than attributed to this PR. The FP8 MoE/MLA add-ons,
normal compiled dense path, and target DeepSeek-V4 path all pass on the final
candidate.
musa_sync verifystill reports the pre-existing FlashAttention cat-4amodified-copy tripwire drift; the complete ordered patch series itself replays
cleanly, and this PR does not change that copied module.
Review order
migration.
v0.24.0-devif GitHub does not do so automatically.