Skip to content

refactor(musa): centralize Qwen and DeepSeek-V4 optimization contracts - #166

Merged
yeahdongcn merged 24 commits into
v0.24.0-devfrom
feat/musa-60083-qwen-contract
Aug 4, 2026
Merged

yeahdongcn merged 24 commits into
v0.24.0-devfrom
feat/musa-60083-qwen-contract

Conversation

@yeahdongcn

@yeahdongcn yeahdongcn commented Aug 3, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR centralizes the MUSA model/execution optimization contract for the
Qwen family and DeepSeek-V4 in one stack.

  • adds an immutable, family-agnostic contract and provider registry;
  • migrates Qwen2/Qwen3/Qwen3.5-Qwen3.6 dense, MoE, GDN, and CosyVoice talker
    selection away from scattered model-name/env gates;
  • adds an exact DeepSeek-V4 provider and binds its shared-MLP, sparse-indexer,
    FLASHMLA, custom-all-reduce, RMSNorm, and GEMV policies;
  • keeps request/shape/stride/capture/provider checks at their owning consumers;
  • fails closed for incomplete or unsupported metadata and preserves the old
    path for non-matching models.

The branch is based on v0.24.0-dev after #156 and ends at
1ff66250d461f4ea208c569131af72d778017ade (with the DSV4 base at
4d0572f12c450c24a9bbfc3c7e2b4fae306a9cb9).

Review follow-up (1ff66250d)

  • removed model-specific _is_*, routed-expert, and DSV4 page-size predicates
    from platform.py;
  • moved those queries into optimization_contract.policy and updated the
    compilation/patch consumers to query policy directly;
  • renamed the Qwen constants to QWEN35_36_*;
  • added explicit Qwen3.5 and Qwen3.6 HF-schema tests plus a fail-closed test
    for a future distinct qwen3_6_* schema.

Contract design

Static model facts (ModelSignature) and runtime facts (ExecutionSignature)
are resolved once into a frozen contract. Providers expose supported and
preferred features; consumers bind the contract to their owning model/module.
Kernel tuning values that are selected by the contract are passed explicitly
where possible (for example, the DSV4 GEMV tile), while dynamic shape and graph
guards remain local.

Explicit generic environment overrides remain available for rollback and A/B
experiments; they no longer provide the default model identity decision.

DeepSeek-V4 performance preservation

The DSV4 work is a refactor on top of #156, not a second claim of the original
fusion uplift. The gate is that contract selection preserves the #156 path.

The exact TP8 workload was measured on one isolated 8 x S5000 node with the
5.2.0-server driver and image
registry.mthreads.com/mcconline/inference/vllm:v0.24.0-ph1-5.2.0-torch2.9.1.post1-20260725:

  • DeepSeek-V4-Flash-Base; BF16 model; block-FP8 weights; FP8 KV;
    FLASHMLA;
  • TP8; max model length 6144; max batched tokens 8195; max sequences 1;
  • prefix cache off; async scheduling on; compilation NONE;
    FULL_DECODE_ONLY, capture sizes [1], copy inputs off;
  • random 4096 input / 1024 output tokens; five warmups;
    A1 -> B1 -> B2 -> A2.
Variant Output TPS TTFT TPOT E2E
Current control (includes #156) 50.8624 792.99 ms 18.8067 ms 20032.20 ms
Contract candidate 50.8604 789.75 ms 18.8105 ms 20032.94 ms
Candidate vs control -0.004% -0.409% +0.021% +0.004%

The #156 controlled
candidate was 51.0648 tok/s, 779.92 ms TTFT, and 18.7415 ms TPOT. The new
candidate differs by -0.40%, +1.26%, and +0.37%, respectively, within a 3%
repeatability envelope. All four measured requests completed with zero
failures and 1024 output tokens.

Validation

Unit, patch, and kernel tests

  • contract/patch regression: 171 passed, 2 skipped;
  • DeepSeek-V4 tests: 112 passed;
  • JIT C++ regression: 64 passed;
  • RMSNorm/SwiGLU numeric and graph tests: 17 passed;
  • GEMV numeric and graph replay: 6 passed;
  • all 104 ordered patches replayed cleanly on pinned vLLM
    ee0da84ab9e04ac7610e28580af62c365e898389.
  • final candidate focused regression on the 5.2.0-server image:
    178 passed, 2 skipped; the two graph-rewrite failures reproduce unchanged
    on the unmodified candidate image and are pre-existing compile/cache drift.

Final-SHA model smokes

  • Qwen3-8B BF16: normal VLLM_COMPILE + FULL_DECODE_ONLY graph,
    FLASH_ATTN, semantic request passed;
  • Qwen3-8B-FP8: FLASH_ATTN semantic request passed;
  • Qwen3-30B-A3B-FP8: FLASH_ATTN/DeepGEMM MoE semantic request passed;
  • DeepSeek-V2-Lite-Chat-FP8: FLASHMLA, six sequential multi-turn requests,
    all HTTP 200, final health passed;
  • DeepSeek-V4-Flash-Base: TP8 FLASHMLA semantic request passed, graph capture
    completed, sparse page 256 and GEMV 16x8 paths observed.

The BF16 MoE/MLA fallback smoke stops on
UnquantizedFusedMoEMethod.is_monolithic; the same exception reproduces with
the pre-DSV4 control image, so it is an existing stack baseline issue rather
than a regression from this PR.

Review and merge

This PR now contains both the Qwen and DeepSeek-V4 changes. The former stacked
DSV4 PR (#167) is now an automatically merged/empty consolidation branch.
Review this PR as one contract change; no PR ordering is required after this
fast-forward.

Evidence bundles:

  • generated/musa60083-dsv4-contract/musa60083-dsv4-abba52-final.tgz
  • generated/musa60083-dsv4-contract/musa60083-dsv4-regression.tgz

@yeahdongcn yeahdongcn changed the title refactor(musa): introduce Qwen optimization contracts refactor(musa): centralize Qwen and DeepSeek-V4 optimization contracts Aug 4, 2026
@yeahdongcn

Copy link
Copy Markdown
Collaborator Author

Addressed the two review comments in commit 1ff66250d461f4ea208c569131af72d778017ade: model-specific _is_*, routed-expert, and DSV4 page-size predicates are no longer implemented in platform.py; callers use optimization_contract.policy. Qwen3.5/3.6 now use explicitly named QWEN35_36_* constants, with positive schema tests for both released forms and a fail-closed test for a future distinct qwen3_6_* schema. Remote focused regression: 178 passed / 2 skipped; the two graph rewrite failures reproduce on the unmodified image baseline.

@yeahdongcn
yeahdongcn marked this pull request as ready for review August 4, 2026 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant