deps: upgrade MUSA torch stack to 2.11 and torchada to 0.1.79

[yeahdongcn]

Summary

Upgrade the MUSA dependency stack while keeping MATE pinned at 0.2.4:

package	torch 2.9 control	torch 2.11 candidate
torch / torch_musa	2.9.1.post1+musa5.2.0	2.11.0.post1+musa5.2.0
torchvision	0.24.1.post1+musa5.2.0	0.26.0.post1+musa5.2.0
torchaudio	2.9.1+musa5.2.0	2.11.0+musa5.2.0
torchada	0.1.77	0.1.79
DeepEP	1.1.0+musa5.2.0torch2.9.1.post1s5000	1.1.0+musa5.2.0torch2.11.0.post1
MATE	0.2.4	0.2.4

The dependency and model-correctness gates are healthy, and the FP8/MoE/TP>1
performance paths tested here are neutral-to-positive. The two BF16 TP1 graph
regressions found during the campaign are now root-caused and fixed:

• plain Qwen3-8B: accept torch 2.11's hoisted opaque FX LayerName and preserve
  the QK RMSNorm + RoPE + KV-cache fusion;
• Qwen3-VL-8B: preserve the torch 2.9 one-dimensional Inductor layout for the
  affected decode copy/index-select/slice/split kernel, scoped to torch >= 2.11
  Qwen3-VL and preserving an explicit user override.

The final code-driven same-card A/B is neutral for both paths, and the final
isolated full-source dual-stack UT sweep is green. This PR is ready for review.

Validation summary

model / path	configuration	correctness	candidate performance vs control	verdict
Unfiltered UT/operator suite	exact full source + both fixes	both: 925 passed, 3 skipped, 0 deselected from 928 collected	not a perf test	Pass
LayerName matcher targeted tests	both dependency stacks	6 passed on each stack	not a perf test	Pass
Platform + Qwen3 targeted tests	both dependency stacks	27 passed on each stack	not a perf test	Pass
Installed dependency gate	exact package imports/versions	passed on both stacks	not a perf test	Pass
Qwen3.5-2B BF16 version isolate	TP1, PIECEWISE, graph partition off, same GPU/driver	exact semantic tokens and 1024-token outputs	BS1 -0.279%; BS4 -0.078%	Neutral
torchvision / torchaudio device ops	transforms, NMS, Resample, MelSpectrogram	numerical/device parity passed	not a perf test	Pass
Qwen2.5-Omni-3B audio	thinker-only, real 16 kHz WAV	identical text/tokens, 3/3 per stack	not measured	Pass correctness
Qwen3.5-27B-FP8	TP2, FULL_AND_PIECEWISE, BS1/4/16/64	exact 1024-token outputs and matching tokens	-0.148% to +0.242%	Neutral
Qwen3.5-35B-A3B-BF16	TP4 MoE, FULL_AND_PIECEWISE, BS1/4/16/64	exact 1024-token outputs and matching tokens	-0.131% to +0.338%	Neutral
Qwen3-32B FP8	TP4, FULL_DECODE_ONLY, BS1/4/16/64	exact 1024-token outputs and matching tokens	-0.315% to +0.740%	Neutral
DeepSeek-V2-Lite-Chat FP8	TP2 MoE, FLASHMLA, FULL_DECODE_ONLY	exact 1024-token outputs and matching tokens	BS1 adjacent repeat +0.174%; BS4/16/64 within 0.4%	Neutral
DeepSeek-V4-Flash-Base	TP8, FLASHMLA, FP8 KV, FULL_DECODE_ONLY	3/3 candidate runs passed	-0.194% vs bracketed control	Neutral
MiniMax-M2.5 FP8	TP8 + EP8 + Eagle3, FULL, BS1/64	exact outputs and matching semantic digest	BS1 +1.104%; BS64 -0.299% mean / +0.433% median	Neutral-to-positive
Qwen3-VL-8B text	BF16 TP1, FULL_DECODE_ONLY, BS1 4k/1k	matching semantic outputs	final code A/B -0.019%	Fixed / neutral
Qwen3-VL-8B multimodal	BF16 TP1, real-image correctness plus random-image perf	exact Red Blue answer; matching tokens	pre-fix regression root-caused to the same decode kernel	Fixed code path
Qwen3-VL-8B no graph	BF16 TP1, BS1 diagnostic	matching semantic output	+3.869%	Diagnostic: graph-specific signal
Qwen3-8B dense, unfixed	BF16 TP1, FULL_DECODE_ONLY, BS1	matching semantic output/tokens	torch 2.11/torchada 0.1.79 -2.117% vs torch 2.9/same torchada	Root-caused
Qwen3-8B dense, matcher fix	BF16 TP1, FULL_DECODE_ONLY, BS1	matching semantic output/tokens; targeted tests pass	+2.139% vs unfixed; -0.024% vs torch 2.9/torchada 0.1.79	Fixed / neutral
Qwen3-8B no graph	BF16 TP1, BS1, earlier diagnostic	matching semantic output/tokens	-1.006%	Historical diagnostic baseline

Same-source A/B setup

• Base: v0.24.0-dev@f45e39e850ef.
• Final candidate head: c5f1997fa; pre-fix model-matrix candidate head:
  b187b3a7b; torch 2.9 control head: b482b89f4.
• The two pre-fix matrix heads differ only in docker/musa.Dockerfile,
  requirements/common.txt, and requirements/musa_private.txt.
• Model-runtime images are based on candidate 66a61c4e0 and control
  11b39b269; exact source overlays provide the final dependency contracts and
  the two Python graph compatibility fixes.
• Broad model matrices: same isolated host, 8 x MTT S5000, driver
  5.2.0-server.
• Qwen3-8B root-cause/fix isolate: same 10.20.34.84 host and MTT S5000 GPU0,
  driver 3.3.5, with identical model, 4096/1024 shape, BS1, graph mode,
  warmup, and five-repetition measurement protocol. The stack/env ablations
  used the same application source; the final row changes only the matcher.
• Qwen3.5-2B version isolate: same 10.20.34.83 host and MTT S5000 GPU0,
  driver 3.3.5-server, exact c5f1997fa overlay, PIECEWISE, and graph
  partition disabled on both stacks.
• Performance legs used normal compiled/captured serving paths, a complete
  warmup, and fresh server processes.

The release-image gate imports torchada first, then imports the complete
runtime set. It requires exact versions for torchada, torch, torch_musa,
torchvision, torchaudio, and DeepEP, and checks the remaining MATE,
flash_attn_3, flash_mla, DeepGEMM, TileLang, Triton, vLLM, and vllm_musa
imports.

Unfiltered unit/operator validation

sweep	stack	result	status
pre-fix broad	torch 2.9 control	919 passed, 3 skipped, 0 deselected in 185.82s	Pass
pre-fix broad	torch 2.11 candidate	919 passed, 3 skipped, 0 deselected in 186.20s	Pass
matcher targeted	torch 2.9 control	6 passed	Pass
matcher targeted	torch 2.11 candidate	6 passed	Pass
final isolated exact-source broad	torch 2.9 control	925 passed, 3 skipped, 0 deselected in 187.75s	Pass
final isolated exact-source broad	torch 2.11 candidate	925 passed, 3 skipped, 0 deselected in 188.01s	Pass

The dependency contract has separate torch 2.9 (torchada 0.1.77,
torchvision 0.24.1) and torch 2.11 (torchada 0.1.79, torchvision 0.26.0)
profiles, so both A/B legs execute the same 928-test inventory. The identical
three skips are one DeepSeek source-Git
check when the installed image has no Git metadata and two legacy
FlashAttention cache checks whose PyO3/safetensors prerequisite is absent.
They are not torchada or torchvision deselections.

The upgrade also exposed and fixed compatibility/test gaps in gated-QKV
Triton scalars, functionalized RMS/SiLU DeepGEMM matching, compiler/ccache test
isolation, stale source/patch contracts, and the release-image dependency gate.

Correctness coverage

path	control / candidate result
Qwen3-32B FP8, DeepSeek-V2-Lite FP8, Qwen3.5 dense/MoE	all measured requests completed; exact 1024-token outputs and matching semantic tokens
DeepSeek-V4 TP8	candidate 3/3 passed; semantic serving checks passed
MiniMax-M2.5 TP8/EP8/Eagle3	identical deterministic France answer, semantic token digest, and exact 1024-token outputs
Qwen3-VL-8B real image	Fast processor selected the torchvision backend on both stacks; a half-red/half-blue image returned exact Red Blue and identical tokens in all three repetitions
torchvision device ops	transform shape [3, 4, 4], checksum -24.094114303588867; MUSA NMS indices [0, 2] on both stacks
torchaudio device ops	Resample shape [1, 8000], MUSA/CPU max diff 0; MelSpectrogram shape [1, 40, 51], max diff 0.001953125
Qwen2.5-Omni-3B audio	real synthetic 16 kHz 440 Hz WAV; exact This audio is a steady tone. and identical eight-token sequence, 3/3 per stack

The Qwen2.5-Omni result covers the audio tower and decoder; it is not a claim
for full Talker/Code2Wav correctness or performance.

Detailed performance tables

All throughput values are steady-state output tokens/s. Dashes mean that an
absolute value was not retained in the consolidated report; no value is
inferred from a percentage.

Qwen3.5-27B-FP8 TP2

Exact 4096 input / 1024 output, FULL_AND_PIECEWISE, three measured
repetitions after warmup.

BS	control tok/s	candidate tok/s	delta	TTFT ms (control / candidate)	TPOT ms (control / candidate)
1	49.834	49.955	+0.242%	446.62 / 442.47	19.649 / 19.605
4	180.135	180.309	+0.096%	1274.60 / 1279.92	20.961 / 20.934
16	525.937	525.159	-0.148%	3985.07 / 3994.11	26.476 / 26.512
64	981.811	982.580	+0.078%	14360.65 / 14318.62	50.747 / 50.738

Verdict: neutral. Evidence: generated/MUSA-60085/qwen35-rebased-f45/.

Qwen3.5-35B-A3B-BF16 TP4 MoE

Exact 4096 input / 1024 output, FULL_AND_PIECEWISE, three measured
repetitions after warmup.

BS	control tok/s	candidate tok/s	delta	TTFT ms (control / candidate)	TPOT ms (control / candidate)
1	107.600	107.564	-0.033%	132.17 / 141.45	9.173 / 9.167
4	361.478	361.955	+0.132%	332.20 / 337.61	10.741 / 10.720
16	912.252	911.061	-0.131%	954.93 / 958.40	16.571 / 16.590
64	2181.809	2189.185	+0.338%	3387.64 / 3367.96	25.747 / 25.668

Verdict: neutral. Evidence: generated/MUSA-60085/qwen35-rebased-f45/.

Qwen3.5-2B BF16 TP1 same-host version isolate

This targeted PIECEWISE run holds source, model, host, GPU, driver, backend,
capture sizes, and graph-partition-off configuration fixed to separate the
dependency-stack effect from a later experimental graph-partition switch.
Exact 4096 input / 1024 output, three BS1 repetitions and the two steady BS4
repetitions after first-concurrency warmup:

BS	torch 2.9 control tok/s	torch 2.11 candidate tok/s	delta	TTFT ms (control / candidate)	TPOT ms (control / candidate)
1	102.6478	102.3612	-0.279%	80.738 / 82.686	9.6723 / 9.6978
4	392.2459	391.9388	-0.078%	209.775 / 211.838	9.9946 / 10.0006

Both stacks returned exact answer 4, matching token receipts, zero request
failures, and full output lengths. Verdict: neutral. The experimental
use_inductor_graph_partition flag is not enabled by this PR and is not used
for the version verdict. Same-GPU evidence:
generated/MUSA-60085/round3-20260809/.

Qwen3-32B FP8 TP4

Exact 4096 input / 1024 output, FULL_DECODE_ONLY, three measured
repetitions after warmup.

BS	control tok/s	candidate tok/s	delta	TTFT ms (control / candidate)	TPOT ms (control / candidate)
1	60.288	60.098	-0.315%	294.90 / 295.80	16.316 / 16.367
4	211.345	210.752	-0.281%	828.48 / 828.82	18.121 / 18.171
16	591.072	593.912	+0.480%	2634.62 / 2650.70	24.458 / 24.311
64	1179.545	1188.268	+0.740%	9901.96 / 9901.52	44.271 / 43.877

Verdict: neutral. Evidence:
generated/MUSA-60085/weekend-expanded/qwen3-32b-fp8/.

DeepSeek-V2-Lite-Chat FP8 TP2

Exact 4096 input / 1024 output, FLASHMLA, FULL_DECODE_ONLY.

BS / run	control tok/s	candidate tok/s	delta	TTFT ms (control / candidate)	TPOT ms (control / candidate)
1, initial ordering	101.138	98.715	-2.396%	146.89 / 152.71	9.753 / 9.990
1, adjacent 5-rep repeat	99.196	99.368	+0.174%	—	—
4	242.787	241.883	-0.372%	401.53 / 393.08	16.083 / 16.153
16	741.036	739.010	-0.273%	1200.79 / 1199.93	20.375 / 20.435
64	1879.266	1875.259	-0.213%	4185.42 / 4190.56	29.675 / 29.741

The adjacent repeat resolves the initial BS1 result as time-order variance.
Verdict: neutral. Evidence:
generated/MUSA-60085/weekend-expanded/deepseek-v2-lite-fp8/.

DeepSeek-V4-Flash-Base TP8

PR #156-aligned BF16/FLASHMLA/FP8-KV recipe, exact 4096 input / 1024 output,
FULL_DECODE_ONLY.

sample	output tok/s	TTFT ms	TPOT ms
control immediately before candidate	51.049	779.70	18.748
candidate mean, 3/3 passed	50.989	785.13	18.765
control immediately after candidate	51.127	784.44	18.713
bracketed control mean	51.088	782.07	18.730
PR #156 latest mean, cross-time reference	51.134	780.88	18.714

Candidate versus bracketed same-host control: throughput -0.194%, TTFT
+0.391%, TPOT +0.186%. Verdict: neutral. One initial control-only launch
timed out alongside a device watchdog reset; clean pre/post controls and all
candidate repetitions passed, so it is retained as availability evidence but
is not attributed to torch 2.11.

Evidence: generated/MUSA-60085/dsv4-rebased-f45/.

MiniMax-M2.5 TP8 / EP8 / Eagle3

Exact 4096 input / 1024 output, five-token Eagle3 speculation, production-style
FULL graphs with capture sizes [6, 24, 96, 384], three measured
repetitions.

BS	control mean tok/s	candidate mean tok/s	mean delta	median tok/s (control / candidate)	latency note
1	73.092	73.899	+1.104%	73.097 / 73.894	stable
64	577.102	575.376	-0.299%	570.635 / 573.107 (+0.433%)	mean TTFT 10703.31 -> 10658.90 ms; TPOT 61.823 -> 61.103 ms

Verdict: neutral-to-positive; BS64 distributions overlap. A failed control
preflight at gpu_memory_utilization=0.95 is excluded; both valid legs used
0.92. Evidence:
generated/MUSA-60085/weekend-expanded/minimax-m25/.

Qwen3-VL-8B BF16 TP1

Normal compiled FULL_DECODE_ONLY path. The initial full matrix retained
deltas for every completed shape:

BS	text 4096/1024 delta	random multimodal 512x512, 256/128 delta
1	-1.51%	-1.70%
4	-2.18%	-1.91%
16	-1.29%	-1.16%
64	-0.59%	not run

Directly adjacent five-repetition repeats confirmed the effect:

workload	BS	control tok/s	candidate tok/s	delta
text	1	66.1444	65.1622	-1.485%
text	4	226.81797	222.5930	-1.863%
random multimodal	1	66.7368	65.9525	-1.175%
random multimodal	4	248.2611	243.1675	-2.052%

The BS4 medians exclude one compile/cold outlier in each leg. These are the
pre-fix discovery results. Evidence:
generated/MUSA-60085/weekend-expanded/qwen3-vl-8b/.

The BS1 no-cudagraph counterfactual reverses the result:

mode	statistic	control tok/s	candidate tok/s	delta
FULL_DECODE_ONLY	adjacent 5-rep median	66.1444	65.1622	-1.485%
no graph	5-rep mean	41.2692	42.8661	+3.869%

Absolute graph/no-graph throughput is not compared as a production claim; the
A/B direction change is the diagnostic signal. Evidence:
generated/MUSA-60085/weekend-expanded/qwen3-vl-nograph/.

MUPTI then localized 77% of the device-time delta to one graph-internal
Inductor pointwise kernel. Counts and topology were identical; torch 2.11 had
changed the torch 2.9 flat 1D grid into a slower 2D grid:

root-cause metric	torch 2.9	torch 2.11 before fix	final behavior
graph-internal kernel records	135,354	135,354	unchanged topology
total MUPTI kernel time	4,055.774 ms	4,149.885 ms	causal delta removed by scoped setting
triton_poi_fused_clone_copy_index_select_slice_split_1 mean	7.032 us	14.930 us	7.140 us with one-dimensional tiling

triton.prefer_nd_tiling=true retained the slow 2D path. Setting
triton.max_tiles=1 restored the target kernel, so the final code applies that
default only to torch >= 2.11 Qwen3VLForConditionalGeneration and uses
setdefault to preserve an explicit user value. A global torchada monkeypatch
was deliberately avoided.

Final same-card, non-profiler, exact 4096/1024 BS1, five-repetition A/B:

stack	mean output tok/s	median output tok/s	delta
torch 2.9 control	60.7741	60.7439	—
torch 2.11 final code fix	60.7625	60.7299	-0.019%

The deterministic real-image result remains exactly Red Blue with the same
token sequence. Root-cause evidence:
generated/MUSA-60085/weekend-expanded/qwen3vl-mupti-rootcause/,
generated/MUSA-60085/weekend-expanded/qwen3vl-fx-rootcause/, and
generated/MUSA-60085/weekend-expanded/qwen3vl-maxtiles1-final/.

Plain Qwen3-8B BF16 TP1

This removes the vision tower while retaining a BF16 dense TP1 decoder. The
original exact-4096/1024, BS1, five-repetition discovery run was:

mode	control mean tok/s	candidate mean tok/s	delta
FULL_DECODE_ONLY	70.019	68.489	-2.186%
no graph	42.392	41.966	-1.006%

That established a graph-path-sensitive regression but did not identify its
cause. A controlled same-host isolate on 10.20.34.84, MTT S5000 GPU0,
driver 3.3.5, then held the model, exact 4096/1024 shape, BS1, graph mode,
warmup, and five-repetition protocol constant:

torch	torchada / ablation	mean tok/s	comparison
2.9	0.1.77	63.355551	reference
2.9	0.1.79	63.123639	-0.366% vs torch 2.9 / torchada 0.1.77
2.11	0.1.79, unfixed	61.787061	-2.117% vs torch 2.9 / same torchada 0.1.79
2.11	0.1.79, AOT0	61.786714	-0.001% vs unfixed
2.11	0.1.79, VLLM_USE_LAYERNAME=0	63.326826	+2.492% vs unfixed; -0.045% vs torch 2.9 / torchada 0.1.77
2.11	0.1.79, matcher fix with default LayerName	63.108478	+2.139% vs unfixed; -0.024% vs torch 2.9 / same torchada 0.1.79

The first five rows use the same application source and vary only the stated
stack/configuration. The last row adds only the matcher fix. Absolute values
from this host are not compared to the earlier discovery host; only within-run
A/B deltas are used. Verdict: plain Qwen3-8B regression root-caused and
fixed, with post-fix performance neutral against the same-torchada torch 2.9
control.

Earlier discovery evidence:
generated/MUSA-60085/weekend-expanded/qwen3-8b-bf16/ and
generated/MUSA-60085/weekend-expanded/qwen3-8b-bf16-nograph/.

Root cause and fix: plain Qwen3-8B

Torch >=2.11 enables opaque LayerName values and hoists the LayerName into an
FX placeholder. The vllm_musa Qwen3 pre-split matcher accepted only str, so
all 36 decoder layers rejected the intended fusion fail-closed. The custom op
already accepts LayerNameType; the narrow fix makes the matcher accept
str | torch.fx.Node and passes the value through unchanged.

MUPTI kernel proof

A 256-token MUPTI capture shows the exact cost of the 36 fusion misses:

metric	torch 2.9 fused control	torch 2.11 unfixed	delta / interpretation
total kernel launches	106,696	125,128	+18,432 = 36 layers x 2 extra kernels x 256 tokens
QK/rotary/cache work	fused_qk...: 9,216 calls, 66.565 ms	split triton_red + rotary + reshape/cache: about 173.437 ms	about +106.873 ms

This accounts for the end-to-end loss inside the captured decode path: the
graph was executing more kernels because the fusion matcher had declined the
torch 2.11 FX form.

Causal exclusions and workaround selection

experiment	result	conclusion
torch 2.11 AOT0	61.786714 tok/s, -0.001% vs unfixed	AOT setting is not causal
torch 2.11 VLLM_USE_LAYERNAME=0	63.326826 tok/s, +2.492% vs unfixed	disabling LayerName proves the LayerName/matcher mechanism
generic direct MUSA Graph A/B	100-node and 500-node graphs within +/-0.4%	no generic graph-executable slowdown reproduced
torch_musa graph source	Python and C++ graph sources identical across the two stacks	no source-level graph implementation drift found there
torchada graph rotation source	source hashes identical; TORCHADA_GRAPH_ROTATION=0 did not improve throughput	torchada rotation is neither cause nor workaround

VLLM_USE_LAYERNAME=0 is a useful causal workaround, but the code fix is
narrower: it preserves the default LayerName path and updates only the matcher
type acceptance. The custom op already supports that value type.

Fix validation

gate	torch 2.9	torch 2.11
targeted matcher/fusion tests	6 passed	6 passed
targeted platform + Qwen3 tests	27 passed	27 passed
broad final, isolated exact full source	925 passed, 3 skipped	925 passed, 3 skipped
Qwen3-8B 4k/1k BS1, default LayerName	63.123639 tok/s with torchada 0.1.79	63.108478 tok/s (-0.024%)

Resolved Qwen3-VL graph regression

Qwen3-VL did not share the LayerName fusion issue. Its graph topology and
kernel counts were identical, but MUPTI and generated-code diffs isolated one
torch 2.11 Inductor pointwise kernel whose grid changed from flat 1D to slower
2D. The scoped platform default restores that kernel to 7.140 us versus the
torch 2.9 value of 7.032 us. The final code-driven five-repetition 4k/1k A/B is
60.7741 versus 60.7625 tok/s (-0.019%) with exact image semantics.

Decision / follow-up

• Plain Qwen3-8B and Qwen3-VL are no longer blockers: both fixes are narrow,
  model-path-specific, and neutral versus the torch 2.9 control.
• Do not use torchada graph rotation or a global Inductor monkeypatch. The
  selected fixes preserve the default LayerName path and scope the tiling
  setting to torch >= 2.11 Qwen3-VL while respecting user configuration.
• The final isolated 928-test dual-stack sweep is green; mark the PR ready for
  review.
• Preserve the two-version dependency contract; torchada 0.1.79 and
  torchvision 0.26.0 are exercised explicitly rather than replacing the
  torch 2.9 control cases.
• No claim is made for cold-start JIT/build time or full Qwen2.5-Omni
  Talker/Code2Wav performance.

Detailed local evidence is in generated/MUSA-60085/final-report.md and
generated/MUSA-60085/weekend-expanded/.