Repository navigation
refactor(musa): centralize Qwen and DeepSeek-V4 optimization contracts - #166
Merged
Merged
Conversation
added 23 commits
August 3, 2026 16:27
Collaborator
Author
|
Addressed the two review comments in commit |
yeahdongcn
marked this pull request as ready for review
August 4, 2026 08:04
This was referenced Aug 7, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR centralizes the MUSA model/execution optimization contract for the
Qwen family and DeepSeek-V4 in one stack.
selection away from scattered model-name/env gates;
FLASHMLA, custom-all-reduce, RMSNorm, and GEMV policies;
path for non-matching models.
The branch is based on
v0.24.0-devafter #156 and ends at1ff66250d461f4ea208c569131af72d778017ade(with the DSV4 base at4d0572f12c450c24a9bbfc3c7e2b4fae306a9cb9).Review follow-up (
1ff66250d)_is_*, routed-expert, and DSV4 page-size predicatesfrom
platform.py;optimization_contract.policyand updated thecompilation/patch consumers to query policy directly;
QWEN35_36_*;for a future distinct
qwen3_6_*schema.Contract design
Static model facts (
ModelSignature) and runtime facts (ExecutionSignature)are resolved once into a frozen contract. Providers expose supported and
preferred features; consumers bind the contract to their owning model/module.
Kernel tuning values that are selected by the contract are passed explicitly
where possible (for example, the DSV4 GEMV tile), while dynamic shape and graph
guards remain local.
Explicit generic environment overrides remain available for rollback and A/B
experiments; they no longer provide the default model identity decision.
DeepSeek-V4 performance preservation
The DSV4 work is a refactor on top of #156, not a second claim of the original
fusion uplift. The gate is that contract selection preserves the #156 path.
The exact TP8 workload was measured on one isolated 8 x S5000 node with the
5.2.0-server driver and image
registry.mthreads.com/mcconline/inference/vllm:v0.24.0-ph1-5.2.0-torch2.9.1.post1-20260725:FLASHMLA;
NONE;FULL_DECODE_ONLY, capture sizes[1], copy inputs off;A1 -> B1 -> B2 -> A2.
The #156 controlled
candidate was 51.0648 tok/s, 779.92 ms TTFT, and 18.7415 ms TPOT. The new
candidate differs by -0.40%, +1.26%, and +0.37%, respectively, within a 3%
repeatability envelope. All four measured requests completed with zero
failures and 1024 output tokens.
Validation
Unit, patch, and kernel tests
171 passed, 2 skipped;112 passed;64 passed;17 passed;6 passed;ee0da84ab9e04ac7610e28580af62c365e898389.178 passed, 2 skipped; the two graph-rewrite failures reproduce unchangedon the unmodified candidate image and are pre-existing compile/cache drift.
Final-SHA model smokes
VLLM_COMPILE+FULL_DECODE_ONLYgraph,FLASH_ATTN, semantic request passed;
all HTTP 200, final health passed;
completed, sparse page 256 and GEMV 16x8 paths observed.
The BF16 MoE/MLA fallback smoke stops on
UnquantizedFusedMoEMethod.is_monolithic; the same exception reproduces withthe pre-DSV4 control image, so it is an existing stack baseline issue rather
than a regression from this PR.
Review and merge
This PR now contains both the Qwen and DeepSeek-V4 changes. The former stacked
DSV4 PR (#167) is now an automatically merged/empty consolidation branch.
Review this PR as one contract change; no PR ordering is required after this
fast-forward.
Evidence bundles:
generated/musa60083-dsv4-contract/musa60083-dsv4-abba52-final.tgzgenerated/musa60083-dsv4-contract/musa60083-dsv4-regression.tgz