Skip to content

refactor(musa): bind DeepSeek-V4 optimizations to model contract - #167

Merged
yeahdongcn merged 8 commits into
feat/musa-60083-qwen-contractfrom
feat/musa-60083-dsv4-contract
Aug 4, 2026
Merged

yeahdongcn merged 8 commits into
feat/musa-60083-qwen-contractfrom
feat/musa-60083-dsv4-contract

Conversation

@yeahdongcn

@yeahdongcn yeahdongcn commented Aug 3, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR extends the optimization-contract infrastructure from #166 to
DeepSeek-V4. It is intentionally stacked on
feat/musa-60083-qwen-contract; reviewers should review only the eight commits
after e2c012dde.

The contract combines immutable model facts from config.json with runtime
execution facts (parallel topology, KV dtype, attention backend, compilation
mode, CUDA-graph mode, and max sequences). Consumers bind the resolved
contract to their owning module and fail closed when identity or execution
metadata is incomplete.

The DeepSeek-V4 contract now owns selection for:

  • the clamp-aware shared-MLP SwiGLU + group-128 FP8 path from perf(musa): fuse DeepSeek-V4 shared-expert SwiGLU FP8 down-projection #156;
  • the native sparse indexer and materialized prefill indexer;
  • the TP8 FLASHMLA sparse page-size and fused-add RMSNorm block-size choices;
  • the custom-all-reduce graph registered-input guard; and
  • the TP8 fused-MoE GEMV tile request, forwarded as graph-static op arguments.

Unsupported model identities, shapes, quantization layouts, parallel
topologies, or execution modes retain the existing path. Explicit generic
kernel overrides remain available and keep precedence over contract-selected
defaults.

Why

DeepSeek-V4 optimizations previously mixed model-name checks, shape checks,
process-wide environment mutation, and consumer-local assumptions. That made
selection difficult to audit and could leak one model's tuning into another
model loaded by the same process.

This change centralizes the static and runtime contract while leaving the
optimized kernels and vLLM-Omni integration boundary unchanged.

Performance alignment with #156

This stack is based on #156, so it does not claim the original +4.46%
shared-MLP fusion gain again. The performance gate here is preservation of the
#156 result while replacing its runtime selection plumbing.

The exact #156 TP8 workload was repeated on one isolated 8 x S5000 node with
the 5.2.0-server driver and image
registry.mthreads.com/mcconline/inference/vllm:v0.24.0-ph1-5.2.0-torch2.9.1.post1-20260725:

  • /home/dist/models/DeepSeek-V4-Flash-Base, BF16 model, block-FP8 weights;
  • TP8, FLASHMLA, FP8 KV, max model length 6144, max batched tokens 8195;
  • max sequences/concurrency 1, prefix cache off, async scheduling on;
  • compilation NONE, FULL_DECODE_ONLY, capture sizes [1], copy inputs off;
  • random 4096 input / 1024 output tokens, ignore EOS, temperature 0;
  • five warmups before each measured phase, A1 -> B1 -> B2 -> A2.
Variant Output TPS runs Mean output TPS Mean TTFT Mean TPOT Mean E2E
Current control (e2c012dde, includes #156) 50.7941 / 50.9307 50.8624 tok/s 792.99 ms 18.8067 ms 20032.20 ms
Contract candidate (4d0572f12) 50.8215 / 50.8993 50.8604 tok/s 789.75 ms 18.8105 ms 20032.94 ms
Candidate vs current control - -0.004% -0.409% +0.021% +0.004%

All four measurements completed 1/1 requests, generated 1024 tokens, and had
zero failures.

For direct comparison, #156 reported a controlled candidate mean of
51.0648 tok/s, 779.92 ms TTFT, and 18.7415 ms TPOT. This candidate differs by
-0.40% TPS, +1.26% TTFT, and +0.37% TPOT, within a 3% repeatability envelope.
The controlled current-stack A/B is effectively neutral, so the contract
migration does not remove the #156 optimization.

Validation

Candidate source: 4d0572f12c450c24a9bbfc3c7e2b4fae306a9cb9.
Pinned upstream vLLM: ee0da84ab9e04ac7610e28580af62c365e898389.

  • contract, patch, and regression slice: 171 passed, 2 skipped;
  • all DeepSeek-V4 tests: 112 passed;
  • JIT C++ regression: 64 passed;
  • fused-add RMSNorm and clamp-SwiGLU numeric/graph tests: 17 passed;
  • GEMV M=1..5 numeric and graph replay tests: 6 passed;
  • real EngineArgs for DeepSeek-V4-Flash-Base resolved the
    deepseek_v4.tp8_flash_base profile;
  • model config SHA256:
    90c592a2dc666e9352a9bc28387414486f074351c7f57a1028bd528e0d0243bc;
  • all 104 ordered patches replayed on the pinned upstream checkout, including
    the new 0101-MUSA-bind-DeepSeek-V4-optimization-contract.patch;
  • ruff passes for the new contract modules and focused DSV4 tests, and
    git diff --check passes for the complete stack. The existing import-order
    findings in tests/test_patches.py and the lazy DeepGEMM import block are
    unchanged from the stack base.

Model regression on the final candidate SHA

Path Mode / backend Result
Qwen3-8B BF16 VLLM_COMPILE, FULL_DECODE_ONLY, FLASH_ATTN HTTP 200, graph capture complete, Beijing, health 200
Qwen3-8B FP8 eager, FLASH_ATTN HTTP 200, quantization=fp8, Beijing, health 200
Qwen3-30B-A3B FP8 eager, FLASH_ATTN, DeepGEMM MoE HTTP 200, Qwen3MoeForCausalLM, Beijing, health 200
DeepSeek-V2-Lite-Chat FP8 eager, FLASHMLA, native GEMV MoE six sequential multi-turn requests, all HTTP 200, Beijing, final health 200
DeepSeek-V4-Flash-Base TP8, #156 graph configuration, FLASHMLA sparse page 256, GEMV 16x8 eager/capture, HTTP 200, Beijing, health 200

The BF16 MoE/MLA fallback smoke currently stops before serving on
UnquantizedFusedMoEMethod.is_monolithic. The same exception reproduces with
the e2c012dde control image, so it is recorded as a pre-existing stack
baseline failure rather than attributed to this PR. The FP8 MoE/MLA add-ons,
normal compiled dense path, and target DeepSeek-V4 path all pass on the final
candidate.

musa_sync verify still reports the pre-existing FlashAttention cat-4a
modified-copy tripwire drift; the complete ordered patch series itself replays
cleanly, and this PR does not change that copied module.

Review order

  1. Review refactor(musa): centralize Qwen and DeepSeek-V4 optimization contracts #166 first for the generic contract types, resolver, and Qwen
    migration.
  2. Review this PR for the DeepSeek-V4 provider and consumer bindings.
  3. Merge refactor(musa): centralize Qwen and DeepSeek-V4 optimization contracts #166 before this stacked PR, then retarget this PR to
    v0.24.0-dev if GitHub does not do so automatically.

@yeahdongcn
yeahdongcn merged commit 4d0572f into feat/musa-60083-qwen-contract Aug 4, 2026
@yeahdongcn

Copy link
Copy Markdown
Collaborator Author

Consolidated into #166. The Qwen branch was fast-forwarded from e2c012dde to the complete Qwen + DeepSeek-V4 stack at 4d0572f12; no code was rewritten.

@yeahdongcn
yeahdongcn deleted the feat/musa-60083-dsv4-contract branch August 24, 2026 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant