[AMD] Fuse quantized in_proj layers in Qwen3.5

[mqhc2020]

Motivation

In Qwen3.5 GDN layers the input projection is issued as two separate GEMMs, in_proj_qkvz and
in_proj_ba. Both read the same activation, so on checkpoints that quantize both to the same
scheme they can be served by one wider GEMM. Keeping them apart costs an extra kernel launch, an
extra activation-quantization pass and an extra epilogue per GDN layer per decode step — overhead
that is a significant share of decode time at low-to-mid concurrency.

This became actionable with the AMD Qwen3.5 MXFP4+AttnFP8 "V2" line, which quantizes in_proj_ba
to FP8; on V2 all four shards resolve to one scheme and the merge is eligible.

The optimization is AMD-only and default-off. To keep that from becoming a correctness question for
everyone else reading qwen3_5.py, all of it lives in its own module rather than inline in the
shared model file — see below.

Modifications

• New python/sglang/srt/models/qwen3_5_gdn_in_proj_merge.py holds the entire feature: the
  SGLANG_GDN_FUSE_QKVZBA + HIP gate (ENABLED), build(), split_output(),
  stacked_params_mapping() and PACKED_MODULES_MAPPING. build() returns None whenever the
  merge does not apply — flag off, non-HIP, LoRA enabled, or a checkpoint whose four shards
  disagree on a quantization scheme — and the caller then constructs in_proj_qkvz and
  in_proj_ba exactly as before. Reviewers on other platforms can therefore establish that nothing
  changes for them by reading one gate, and the module is dead code off ROCm.

  build() assembles the four shards (qkv, z, b, a) into a single in_proj_qkvzba via
  MergedColumnParallelLinear, padding each rank's N up to the next multiple of 128 for the aiter
  a8w8 tile (correctness only needs N % 16 == 0); the padding rows are zeroed and no checkpoint
  tensor reaches them. A scheme mismatch raises ValueError inside the layer construction, which is
  caught and logged once before falling back. Mixed-precision checkpoints are therefore untouched —
  on V1, where in_proj_a/b stay bf16, the model loads exactly as before.

• python/sglang/srt/models/qwen3_5.py now delegates instead of implementing. create_qkvzba_proj()
  is a thin wrapper over build(), _forward_input_proj() calls split_output(), the four
  load_weights() implementations splice in stacked_params_mapping(self), and
  packed_modules_mapping splices in PACKED_MODULES_MAPPING. That mapping is {} off HIP, so
  non-AMD platforms never see the merged parameter name at all.

• python/sglang/kernels/ops/attention/triton_gdn_fused_proj.py: parameterize the
  split/reshape/cat kernel by QKVZ_STRIDE / BA_STRIDE rather than deriving the row stride from
  logical widths, so it can read column slices of a wider, padded projection. split_output()
  returns views rather than copies, so the alignment padding falls away without touching memory.

• No communicator.py change, and _forward_input_proj_fused_quant_amd is untouched: the merged
  GEMM is dispatched at the top of _forward_input_proj, where _select_fused_ar_input_for_linear
  handles a plain tensor and a fused-AR tuple alike, so one branch covers both entry paths.

• Tests: a GSM8K A/B accuracy gate on MI35x (below), plus a case in the existing decode-fusion
  kernel test covering the strided, offset column views the merge produces.

CI coverage

test/registered/e2e/models/test_qwen35_gdn_in_proj_fusion_mi35x.py runs two TP2 servers in
parallel on one 8-GPU MI35x host (stage-c-test-large-8-gpu-amd-mi35x): the merged arm with
SGLANG_GDN_FUSE_QKVZBA=1 against the default separate-projection arm, and requires the merged arm
to hold GSM8K accuracy. Since the feature is default-off, this is the only CI that executes it at
all.

The fused arm's server log is checked for the fallback message, because a checkpoint whose shards
disagree would silently degrade the test into baseline-vs-baseline and pass. TP=2 rather than TP=4:
TP=4 narrows in_proj_ba to 32 columns, which gemm_a8w8_bpreshuffle has no kernel for once M
reaches 256.

test/registered/kernels/ops/attention/test_gdn_decode_fused_proj_conv.py gains
test_padded_merged_projection_column_views_are_supported, which feeds the decode fusion a padded
merged projection — stride(0) wider than the slice, nonzero storage offset on the ba view — and
requires bitwise equality against densely packed inputs.

Accuracy Tests

gsm8k 5-shot, full 1319 questions, lm_eval local-chat-completions with
--apply_chat_template against /v1/chat/completions, server run with --reasoning-parser qwen3
and --default-chat-template-kwargs '{"enable_thinking": false}'.
Checkpoint: AMD Qwen3.5-397B-A17B-MXFP4-AttnFP8-V2 (PR#4 revision), 2× MI355X, TP=2.

SGLANG_GDN_FUSE_QKVZBA	strict-match	flexible-extract
0 (separate, today's behaviour)	0.9712 ± 0.0046	0.9712 ± 0.0046
1 (merged)	0.9773 ± 0.0041	0.9765 ± 0.0042

The merged arm is 0.6 pp higher, well inside the ±0.4–0.5 pp standard error of each measurement,
i.e. the two arms are statistically indistinguishable. No accuracy regression.

Note for anyone reproducing: Qwen3.5 V2 is a thinking model. Grading it on /v1/completions
scores the raw <think> text and yields a spurious ~0.82–0.83 for both arms; the chat +
thinking-off recipe above is the one that reflects real quality.

Speed Tests and Profiling

bench_serving, random dataset, OSL 1024, --random-range-ratio 0.8, --num-prompts 10×cc,
request_rate=inf with --max-concurrency, seed 0. Same checkpoint/hardware as above. Server:
--attention-backend aiter --kv-cache-dtype fp8_e4m3 --page-size 16 --chunked-prefill-size 8192 --disable-radix-cache --enable-aiter-allreduce-fusion --enable-dense-fp8 (effective
mem_fraction_static 0.578, max_running_requests 636); the 1k regime adds --enable-mixed-chunk.
TPOT figures are medians.

8k context (ISL 8192, OSL 1024)

cc	prompts	TPOT separate (ms)	TPOT merged (ms)	ΔTPOT	total tok/s separate	merged	Δ
4	40	8.84	8.60	−2.71%	3845.6	3954.6	+2.83%
8	80	11.72	11.38	−2.90%	5782.4	5964.7	+3.15%
16	160	16.57	16.15	−2.53%	8299.6	8511.3	+2.55%
32	320	25.34	24.78	−2.21%	10930.6	11136.7	+1.89%
64	640	40.49	39.92	−1.41%	13903.8	14095.0	+1.38%

1k context (ISL 1024, OSL 1024)

cc	prompts	TPOT separate (ms)	TPOT merged (ms)	ΔTPOT	total tok/s separate	merged	Δ
4	40	8.06	7.85	−2.61%	950.1	976.2	+2.75%
8	80	10.11	9.80	−3.07%	1532.4	1579.4	+3.07%
16	160	13.06	12.72	−2.60%	2409.1	2475.6	+2.76%
32	320	17.91	17.52	−2.18%	3498.6	3571.2	+2.07%
64	640	24.81	24.51	−1.21%	5065.3	5132.7	+1.33%

All 20 cells (2 regimes × 5 concurrencies × 2 arms) favour the merged arm on both TPOT and
throughput; tok/s columns are total input+output throughput, and output-only moves by the same
percentage. The gain is largest at low-to-mid concurrency, where the per-GEMM launch and epilogue
overhead removed by the merge is the biggest share of decode time, and shrinks as concurrency grows
and the GEMMs become compute-bound. An independent earlier repeat of this sweep reproduced the same
deltas within a few tenths of a percent.

Absolute numbers vs the previous V2 baseline

For context beyond the A/B, the same sweep against our July 28 V2 baseline (aiter #4017+#3986 on
the published Qwen3.5-397B-A17B-MXFP4-AttnFP8-V2), run with an identical harness and identical
server settings. Medians, matching how that baseline was recorded:

ISL	cc	out tok/s/gpu	tok/s	TTFT (ms)	TPOT (ms)	ITL (ms)
1024	4	234 → 243 (+3.8%)	940 → 976	95.9 → 92.2	8.1 → 7.8	7.9 → 7.6
1024	8	383 → 396 (+3.5%)	1527 → 1579	99.0 → 95.6	10.1 → 9.8	9.6 → 9.3
1024	16	596 → 616 (+3.3%)	2398 → 2476	103.4 → 100.3	13.1 → 12.7	12.0 → 11.7
1024	32	866 → 894 (+3.3%)	3459 → 3571	112.0 → 106.9	18.1 → 17.5	16.0 → 15.6
1024	64	1240 → 1283 (+3.5%)	4962 → 5133	144.7 → 122.2	25.1 → 24.5	21.1 → 20.9
8192	4	214 → 220 (+2.8%)	3840 → 3955	293.8 → 291.4	8.9 → 8.6	8.0 → 7.7
8192	8	326 → 335 (+2.8%)	5804 → 5965	294.4 → 287.8	11.7 → 11.4	9.7 → 9.4
8192	16	460 → 472 (+2.5%)	8294 → 8511	308.4 → 300.4	16.6 → 16.1	12.2 → 11.9
8192	32	612 → 623 (+1.7%)	10942 → 11137	320.4 → 314.4	25.2 → 24.8	16.2 → 16.0
8192	64	775 → 782 (+0.9%)	13970 → 14095	343.9 → 337.4	40.2 → 39.9	21.9 → 21.7

Every cell improves on every metric. Note these are medians; under request_rate=inf the initial
burst gives mean TTFT a long tail (at 8k/cc64 the baseline's mean TTFT is 1358 ms against a median
of 344 ms), so mean and median must not be compared across runs.

Merge engagement was verified, not assumed. At the time these sweeps were taken nothing logged
the merge, so both harnesses used the KV-cache token count as a signature: the padded merged
projection carries ~12 MB more weights per rank, so at a fixed mem-fraction-static the merged arm
has slightly less room for KV. The perf sweep reported 4,505,008 → 4,495,728 tokens and the accuracy
run 11,774,352 → 11,765,072 — the same −9,280 delta in two independently configured runs. Identical
counts would have meant the flag never took effect. The CI gate now checks the fallback log line
directly instead of relying on this signature.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ❌ Run #34545848390
Latest PR Test (Extra): ❌ Run #34545847979
Latest PR Test (AMD ROCm 10): ❌ Run #34545848258

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[mqhc2020]

@amd-bot ci-status

[amd-bot]

@mqhc2020

CI Status for PR #33068

Merge verdict: ❌ Do not merge on this CI run — the signal is incomplete and the PR's own tests were largely not executed. All 12 red X's are the CI rate-limit gate (not code failures): user mqhc2020 triggered these workflows at 10:18:34Z, so the 11:46 push to head bdb5eda4 hit the 120-min per-user cooldown and fast-failed at the gate — cascading to every downstream test job (skipped) and the *-finish aggregators (failed). Zero test jobs executed a failure. Only the ROCm 7.2 (AMD), Lint and MLX pipelines cleared the gate and are still running.

Caution

PR CI is incomplete AND this PR's changed code is not verified. The rate-limit gate fast-failed 8 workflows (Base, Extra, MUSA, NPU, XPU, Xeon, Arm64, Extra-AMD) — those downstream jobs are not tested. The PR's two new PR-CI tests were affected:

• test/registered/ops/test_gdn_fused_proj_amd.py (suite stage-b-test-1-gpu-small-amd) — the direct test of the fused-proj kernel. Blocked on the main AMD path; only pending on the still-running ROCm 7.2 run (stage-b not yet reached).
• test/registered/unit/models/test_qwen3_5_gdn_in_proj_merge.py (suite base-a-test-cpu) — did not run; PR Test Base fast-failed at the gate.
• test/registered/amd/accuracy/mi35x/test_qwen35_eval_mi35x.py is nightly=True → never runs on PR CI regardless.

Green here would prove nothing. After the cooldown expires (~12:18Z, 120 min from 10:18:34Z), re-run / update the branch so the full pipeline executes. For high-priority full signal, the bypass-fastfail label can be used sparingly.

Changed files: python/sglang/kernels/ops/attention/triton_gdn_fused_proj.py (+13/-8), python/sglang/srt/models/qwen3_5.py (+126/-29), test/.../test_qwen35_eval_mi35x.py (+17/-2), test/registered/ops/test_gdn_fused_proj_amd.py (+176 new), test/registered/unit/models/test_qwen3_5_gdn_in_proj_merge.py (+279 new)

Executed CI failure attribution: AMD: 0 executed failures (all gate-blocked or pending) · Others: 0 executed failures (all gate-blocked). All 12 red checks = 1 root cause (rate-limit gate). Still running: Lint, PR Test (MLX), PR Test ROCm 7.2 (AMD) — do not call this run green.

Gate / cascade failures (collapsed to one root cause)

Workflow	Job	Cause	Related to PR?	Why
PR Test Base	call-gate / pr-gate	Rate limit: "already triggered 'PR Test Base' … at 10:18:34Z, wait 120 min"	🟢 No	Gate rate-limit, not code
PR Test Base	pr-test-finish	call-gate: failure → aggregator	🟢 No	Cascade from gate
PR Test (MUSA)	pr-gate	Rate limit: "already triggered 'PR Test (MUSA)' … wait 120 min"	🟢 No	Gate rate-limit
PR Test Extra	call-gate / pr-gate + finish	Rate-limit cascade	🟢 No	Gate rate-limit
PR Test Extra (AMD)	call-gate + finish	Rate-limit cascade	🟢 No	Gate rate-limit
PR Test (Arm64/Xeon/NPU/XPU)	pr-gate (Arm64 · Xeon · NPU · XPU)	Rate-limit cascade	🟢 No	Gate rate-limit

(Verified the rate-limit message directly in the Base and MUSA gate logs; the rest show the same pr-gate failure with no test job reaching execution.)

Details / what to do before merge

• This is a benign gate block, not a broken PR — no code failure exists in this run. But it also means no meaningful test signal, so the ❌'s must not be "overridden and merged."
• Re-trigger after the cooldown (~12:18Z) by pushing an empty commit or re-running the failed workflows, so PR Test Base (covers the new CPU unit test) and the AMD stage-b suites (cover the new op test) actually execute.
• Watch the in-progress ROCm 7.2 run 31697000067: its gate passed and stage-a is running; stage-b-test-1-gpu-small-amd-rocm720 (which includes test_gdn_fused_proj_amd.py) is still pending — confirm it goes green before relying on AMD coverage.
• The mi35x accuracy test is nightly-only — it will not validate this PR on PR CI at all; the author should run it manually (or via nightly) to verify end-to-end Qwen3.5 accuracy with the fused in_proj change.

---

Generated by amd-bot using Claude Code CLI

[mqhc2020]

@amd-bot ci-status

[bingxche]

@mqhc2020

CI Status for PR #33068

Merge verdict: This PR's changed code is well-exercised and green — all three new tests pass, including the mi35x e2e test that explicitly enables the default-off SGLANG_GDN_FUSE_QKVZBA=1 fused path. The 7 executed CI failures are in unrelated code (radix cache, MoE-EP, lean/quark kernels on a broken mi35x runner, a Rust link/infra error, and two numeric determinism/logprob checks) — none touch the Qwen3.5 GDN fusion. The red pr-test-finish / pr-test-amd-finish checks are blocking but reflect that pre-existing noise (the author already applied the bypass-fastfail label). One failure (test_deterministic) is possibly adjacent via the triton_backend.py edit and should be re-run to be safe. Not a clean auto-merge only because of the blocking red aggregators, not because of this PR.

Note

Changed paths ARE exercised by PR CI. The fused feature is default-off (SGLANG_GDN_FUSE_QKVZBA), and its dedicated tests all ran and passed on this head SHA: test_gdn_fused_proj_amd.py (AMD unit, merge/kernel/gating) ✅, test_gdn_decode_fused_proj_conv.py (NVIDIA base-b kernel) ✅, and test_qwen35_gdn_in_proj_fusion_mi35x.py (AMD stage-c e2e, sets SGLANG_GDN_FUSE_QKVZBA=1, 612 s) ✅. Green here genuinely verifies the feature.

Changed files: qwen3_5.py (+154/-36), triton_gdn_fused_proj.py (+16/-7), triton_backend.py (+4/-0), decode_attention.py (+1/-0), plus 3 new test files.

Executed CI failure attribution: AMD: 4 failures (0 related) · Others: 3 failures (0 clearly related, 1 possibly) — *-finish jobs are fast-fail aggregators, collapsed into their root causes below.

AMD Executed Failures

Job	Test File	Test Function	Error	Related?	Why
stage-b-1gpu-small-amd-mi35x	test/registered/kernels/test_lean_attention.py	test_paged_kv_parity	ValueError: unexpected KV buffer ndim=4	🟢	Fails in _extract_kv_strides (decode_attention.py:211) on the enable_lean=False call — a frame the PR does not touch; PR's only edit here is a lean_locks is not None guard on the lean branch
stage-b-1gpu-small-amd-mi35x	test/registered/quant/test_quark_mxfp4.py	(suite)	exit 1	🟢	Quark MXFP4 kernel; same mi35x runner failed 3 unrelated kernel tests → runner/env-wide, not GDN fusion
stage-b-1gpu-small-amd-mi35x	test/registered/unit/layers/attention/test_mla_decode_forced_splits.py	(suite)	exit 1	🟢	MLA decode; unrelated to Qwen3.5 GDN
stage-b-2gpu-large-amd	test/registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_full.py	(suite)	exit 5	🟢	Radix-cache KL test; no overlap with changed files
stage-b-1gpu-small-amd (11)	test/registered/attention/test_deterministic.py	TestTritonDeterministic.test_prefix_with_logprobs	AssertionError (logprobs differ, after retries)	🟡	PR edits triton_backend.py cuda-graph out-graph metadata, but the added lean_Mp/Lp/Op/locks args are lean-only and inert with lean off; test runs Llama-3.2-1B with lean disabled → likely a determinism flake, but re-run to confirm
stage-c-test-large-8-gpu-amd (0)	test/registered/amd/test_moriep_small.py	(suite)	Timed out at 5400 s	🟢	MoRI MoE-EP test; timeout, unrelated to changed code

Other Executed Failures

Job	Test File	Test Function	Error	Related?	Why
base-b-test-1-gpu-small (7)	test/registered/openai_server/basic/test_openai_completion_rust.py	test_logprobs_have_zero_kl_against_python_frontend	AssertionError: Lists differ (logprob KL)	🟢	Rust OpenAI-server logprob parity on Llama-3.2-1B; no Qwen3.5/GDN path, PR touches no Rust
base-c-test-4-gpu-gb300 (0) · (1)	N/A (Rust build)	N/A	ld: cannot find -ltorch_cuda / -ltorch / -lc10 → cargo exit 101	🟢	Torch libs missing from linker path on the gb300 runner — an infra/toolchain build failure; PR changes zero Rust files. gb300 4-GPU tests did not execute (see caution)

Details / what to do before merge

• CI completeness (minor): both base-c-test-4-gpu-gb300 shards failed at the Rust extension build (-ltorch* not found), so those 4-GPU gb300 test partitions never ran. This is a runner/toolchain issue, not this PR, and no changed code path is gb300-specific — but it is an unrelated CI gap. bypass-fastfail is already applied, so downstream stages were not fast-fail-cascaded; the skipped *-extra / multimodal / sgl-kernel / rust-ext jobs are path/gate-filtered (PR touches none of those areas), not fast-fail casualties.
• Only 🟡 item to close: re-run stage-b-1gpu-small-amd (11) (test_deterministic.py::test_prefix_with_logprobs). Expected to pass on retry; if it reproduces, check whether the triton_backend.py out-graph lean buffers are being consumed with lean disabled. All other reds are unrelated pre-existing/infra noise.
• No action needed on coverage — the default-off fusion is directly verified by the passing mi35x e2e test.

---

Generated by amd-bot using Claude Code CLI

[mqhc2020]

@amd-bot ci-status

[bingxche]

@mqhc2020

CI Status for PR #33068

Merge verdict: No executed failure is attributable to this PR, and — most importantly — this PR's changed code is exercised and green on both NVIDIA and AMD (MI300 + MI35x). All 8 executed job failures are in unrelated backends (DeepSeek-V4, hicache, NIXL disagg, radix-cache, MLA/lean-attention, openai-completion) or are pure infra (docker-pull, UCX). However, PR CI is incomplete: the fast-fail cascade skipped every "extra" pipeline (AMD-extra, NVIDIA-extra, NPU base-c) and the NPU base-b jobs were cancelled — so full signal has not been obtained.

Note

The PR's own code paths are covered and all three new tests passed on this head SHA:

• test/registered/ops/test_gdn_fused_proj_amd.py ✓ PASSED — AMD stage-b-test-1-gpu-small-amd (MI300)
• test/registered/amd/perf/mi35x/test_qwen35_gdn_in_proj_fusion_mi35x.py ✓ PASSED (603s) — AMD stage-c-test-large-8-gpu-amd-mi35x (MI35x)
• test/registered/kernels/ops/attention/test_gdn_decode_fused_proj_conv.py ✓ PASSED — NVIDIA base-b 1-gpu-large

Caution

PR CI is incomplete due to fast-fail. Because unrelated stage-b/base-b tests failed, these downstream pipelines were skipped / cancelled and are NOT tested: AMD extra (call-pr-test-amd-extra, extra-*-amd, multimodal-gen-*-amd, sgl-kernel-unit-test-amd), NVIDIA extra (extra-a/b H100/H200/B200, sgl-kernel-build-wheels), and the entire NPU pipeline (base-b-*-npu-a3 cancelled, base-c-*-npu-a3 skipped). The call-gate / pr-gate is RED for the same reason. The PR's changed kernel/model paths don't reach these skipped suites, so this doesn't hide a regression from this PR, but the author should confirm required vendor CI finishes (rerun / update branch, or bypass-fastfail sparingly) before merge.

Changed files: triton_gdn_fused_proj.py (+16/-7), qwen3_5.py (+154/-36), test_qwen35_gdn_in_proj_fusion_mi35x.py (+347), test_gdn_fused_proj_amd.py (+176), test_gdn_decode_fused_proj_conv.py (+67)

Executed CI failure attribution: AMD: 7 failures (0 related) · Others: 1 failure (0 related). Extra/NPU pipelines: skipped/cancelled by fast-fail (not counted).

AMD Executed Failures

Job	Test File	Test Function	Error	Related?	Why
stage-c-dsv4-flash-fp4-fp8-amd-mi35x	test/registered/amd/test_deepseek_v4_flash_fp8.py	setUpClass	ValueError: SGLANG_OPT_FP8_WO_A_GEMM ... wo_a.weight bfloat16, expected float8	🟢	DeepSeek-V4 fp8 checkpoint/env issue; PR only touches Qwen3.5 GDN in_proj
stage-c-dsv4-flash-fp4-fp8-amd-mi35x	test/registered/hicache/test_hicache_storage_umbp_backend.py	N/A	exit code 5	🟢	hicache storage backend, unrelated code path
stage-b-1gpu-small-amd (9)	N/A	N/A	docker pull rocm/sgl-dev:...mi30x-20260904 failed after 6 attempts	🟢	Infra: image pull failure, not code
stage-b-1gpu-small-amd (10)	N/A	N/A	docker pull ... failed after 6 attempts	🟢	Infra: image pull failure
stage-b-1gpu-small-amd (11)	test/registered/attention/test_deterministic.py	—	exit 1; KeyError '/loky-...' (worker crash)	🟢	General attention determinism test; loky worker crash, not GDN fusion
stage-b-2gpu-large-amd (1)	test/registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_full.py	—	exit code 5	🟢	Radix-cache KL test, unrelated
stage-b-large-8-gpu-mi35x-disaggregation-amd	test/registered/amd/disaggregation/test_nixl_transfer_engine_e2e.py	—	nixlBackendError: NIXL_ERR_BACKEND; UCX Destination unreachable; server exit -9	🟢	Disagg NIXL/UCX transport infra, unrelated
stage-b-1gpu-small-amd-mi35x	test/registered/unit/layers/attention/test_mla_decode_forced_splits.py	—	ValueError: unexpected KV buffer ndim=4, shape=[257,64,1,576]	🟢	MLA decode path (+ test_lean_attention.py), unrelated to GDN

Other Executed Failures

Job	Test File	Test Function	Error	Related?	Why
base-b-test-1-gpu-small (3)	test/registered/openai_server/basic/test_openai_completion_rust.py	logprobs check	AssertionError: Lists differ (logprobs)	🟢	openai completion logprobs mismatch (flaky/numeric), no Qwen3.5/GDN path

Details / what to do before merge

• Coverage is satisfied — no action needed to verify the PR's logic; the fused GDN in_proj kernel + Qwen3.5 model changes are exercised and green on NVIDIA base-b, AMD MI300 (test_gdn_fused_proj_amd), and AMD MI35x (test_qwen35_gdn_in_proj_fusion_mi35x).
• Get complete signal before merge: rerun the failed stage-b/base-b shards (docker-pull infra + flakes) or update the branch so the fast-fail cascade clears and the AMD-extra / NVIDIA-extra / NPU pipelines actually execute. Use the bypass-fastfail label only if full downstream signal is needed urgently (it consumes more CI).
• The unrelated failures (DeepSeek-V4 fp8 checkpoint, NIXL disagg, deterministic loky crash, radix-cache KL, MLA decode, openai logprobs) look pre-existing/flaky on main — they are for the CI monitor, not this PR author.

---

Generated by amd-bot using Claude Code CLI

[sogalin]

@yuan-luo, could you help review this PR, thanks.

[mqhc2020]

@amd-bot ci-status

[bingxche]

@mqhc2020

CI Status for PR #33068

Merge verdict: The PR's own changes are well-covered and green — the MI35X fused-path e2e gate and the NVIDIA kernel column-view test both ran and passed on this head SHA. All 15 executed failures are in unrelated subsystems (mem cache, radix cache, disaggregation, MoE-EP, DeepSeek-V4, quark-mxfp4, NPU perf/diffusion, MLX, CPU); none touch the Qwen3.5 GDN in_proj code, and the new fusion is off by default + HIP-gated, so it is inert everywhere it wasn't explicitly enabled. CI is complete — the bypass-fastfail label forced every stage to run despite failures (1 XPU job still queued). Not safe to merge on a green button alone only because the many unrelated reds should be confirmed as known-flaky-on-main first — but there is no PR-attributable failure.

Note

This PR's changed code is exercised by PR CI and passes. The fused path (SGLANG_GDN_FUSE_QKVZBA=1, HIP-only) is verified by test_qwen35_gdn_in_proj_fusion_mi35x.py → passed (635s) in stage-c-test-large-8-gpu-amd-mi35x, and the padded merged-projection kernel views by test_gdn_decode_fused_proj_conv.py → passed (6s) in base-b-test-1-gpu-large. Green here genuinely verifies the feature.

Changed files: qwen3_5.py (+74/-29), qwen3_5_gdn_in_proj_merge.py (new +144), triton_gdn_fused_proj.py (+16/-7), test_qwen35_gdn_in_proj_fusion_mi35x.py (new +342), test_gdn_decode_fused_proj_conv.py (+67).

Executed CI failure attribution: AMD: 8 job-failures (0 related) · Others: 5 failures (0 related) · plus 9 *-finish/gate cascade jobs (fail because upstream stages had reds — not independent) · 1 XPU job stage-b-test-1-gpu-xpu still queued (do not treat as passed).

AMD Executed Failures

Job	Test File	Test Function	Error	Related?	Why
stage-a-1gpu-small	test/registered/unit/mem_cache/test_umbp_store.py	(suite)	exit 1	🟢	Memory-cache store; PR touches none of it
stage-b-1gpu-small (11)	test/registered/attention/test_deterministic.py	(suite)	exit 1	🟢	Attention determinism; not GDN in_proj
stage-b-2gpu-large (1)	test/registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_full.py	(suite)	exit 5	🟢	Radix-cache KL; unrelated path
stage-b-1gpu-small-mi35x	test/registered/quant/test_quark_mxfp4.py	(accuracy)	AssertionError: 0.846 not greater than 0.85	🟢	Quark MXFP4 accuracy, borderline threshold miss; independent of GDN
stage-b-8gpu-mi35x-disagg	test/registered/amd/disaggregation/test_nixl_transfer_engine_e2e.py, …/test_disaggregation_pp.py	(server)	nixlBackendError: NIXL_ERR_BACKEND, server exit -9	🟢	NIXL disaggregation transport; unrelated
stage-c-large-8gpu (0-3)	test/registered/amd/test_moriep_small.py	(server)	AttributeError: 'Tensor' object has no attribute 'format_ue8m0'	🟢	MoE-EP path; 4 shards = 1 cluster; unrelated
stage-c-dsv4-flash-fp4-fp8-mi35x	test/registered/amd/test_deepseek_v4_flash_fp4.py (+fp8, +fp8_tbo)	(server)	ValueError: SGLANG_OPT_FP8_WO_A_GEMM … wo_a.weight has dtype bf16	🟢	DeepSeek-V4 fp8 GEMM env issue; unrelated model
stage-c-dsv4-pro-fp4-mi35x	test/registered/amd/test_deepseek_v4_pro_fp4.py	(server)	AssertionError (exit 5)	🟢	DeepSeek-V4; unrelated model

Other Executed Failures

Job	Test File	Test Function	Error	Related?	Why
base-c-test-perf-16-npu-a3	test/registered/npu/performance/qwen3_5_397b/test_npu_qwen3_5_397b_w4a8_8p_in3k5_out1k5_50ms.py	(perf)	AssertionError: 99.93 not <= 51.0	🟡→🟢	Only red that names Qwen3.5, but NPU is non-HIP ⇒ build() returns None, in_proj_qkvz/ba built exactly as before, _SEPARATE_STACKED_MAPPING unchanged — the changed code is inert on NPU; a ~2× latency miss can't come from inert code ⇒ infra/perf-gate flake
base-a-test-cpu (6)	test/registered/unit/multimodal/test_gpu_feature_transport.py	(suite)	AttributeError: 'Scheduler' object has no attribute 'disable_radix_cache'	🟢	CPU scheduler attr; unrelated
base-b-test-2-gpu-large (0)	test/registered/disaggregation/test_disaggregation_basic.py	(server)	timeout after 1200s	🟢	Disaggregation E2E timeout; unrelated
multimodal-gen-test-4-npu-a3 (0)	.../ascend/test_server_2_npu.py	test_diffusion_generation[wan2_2_t2v_14b_w8a8_2npu]	diffusion check failed	🟢	Diffusion (WAN 2.2) product; unrelated
stage-a-unit-test-mlx	test/registered/unit/hardware_backend/mlx/test_scheduler_mixin.py	(suite)	exit 1	🟢	MLX/arm64 backend; unrelated

Cascade only (not independent failures): pr-test-amd-finish, pr-test-npu-finish, pr-test-finish, pr-test-mlx-finish, pr-test-extra-finish, pr-test-amd-extra-finish, and the pr-gate/call-gate jobs all report failure solely because upstream stages had reds.

Details / what to do before merge

• Coverage is satisfied — no action needed to verify the feature itself; the fused MI35X path and the kernel view path both ran green on this SHA.
• Confirm the reds are pre-existing, not this PR: the failing families (radix cache, disaggregation, MoE-EP format_ue8m0, DeepSeek-V4 fp8/fp4, quark-mxfp4 accuracy, NPU 397B perf, WAN diffusion, MLX scheduler) are all outside the diff. Recommend a quick check that the same jobs are red on recent main runs (that's the CI-monitor's territory, not this check) before merging.
• The one to eyeball: the NPU Qwen3.5-397B 50ms perf gate — it's name-adjacent to the diff. Confirmed harmless by code path (fusion disabled on non-HIP), but if you want zero doubt, re-run that single NPU perf job; a pass confirms flake.
• Pending: wait for stage-b-test-1-gpu-xpu to conclude before final merge — it is still queued, so XPU signal is incomplete.

---

Generated by amd-bot using Claude Code CLI

[mqhc2020]

@amd-bot ci-status

[bingxche]

@mqhc2020

CI Status for PR #33068

Merge verdict: Do not merge yet — PR CI is incomplete. The entire NVIDIA pipeline (pr-test, pr-test-extra) never ran: check-changes hard-failed on a rebase/maintenance gate (the PR is diverged from required base 3700c4ee), so all 24 downstream NVIDIA jobs were skipped. AMD Cit did run: every executed AMD failure is in a test unrelated to this PR (wave-attention, LoRA, deterministic, MoRI-EP, DeepSeek-V4, NIXL), and the crash cluster is an OOM/-9 infra pattern on the shared MI35x runners — none is attributable to the Qwen3.5 GDN in_proj change. The feature's own accuracy gate passed (see caution below).

Caution

PR CI is incomplete and the kernel-level change is unverified. check-changes fast-failed the maintenance rebase gate, so all pr-test / pr-test-extra (NVIDIA) jobs were skipped — including base-b, where the PR's modified kernel test test/registered/kernels/ops/attention/test_gdn_decode_fused_proj_conv.py (registered register_cuda_ci(stage="base-b"), +67 lines) runs. That code path was not exercised by any run on this head SHA. Author must rebase onto latest main (base 3700c4ee) and re-push to get full NVIDIA signal. pr-test-amd-extra also didn't run — its call-gate failed only because the run-ci-extra label is missing (benign; add it if extra AMD coverage is wanted).

Note

The core feature is covered on AMD and passed: the new e2e gate test/registered/e2e/models/test_qwen35_gdn_in_proj_fusion_mi35x.py (suite stage-c-test-large-8-gpu-amd-mi35x) ran on shard 1 and passed (elapsed=644s), validating merged (SGLANG_GDN_FUSE_QKVZBA=1) vs. separate in_proj accuracy on MI35x — this also exercises the default (non-merged) Qwen3.5 path.

Changed files: qwen3_5.py (+16/-17), qwen3_5_gdn_in_proj_merge.py (new, +216), test_qwen35_gdn_in_proj_fusion_mi35x.py (new, +342), test_gdn_decode_fused_proj_conv.py (+67).

Executed CI failure attribution: AMD: 11 failures (0 related) · Others (NPU/MLX/CPU): 4 failures (0 related) · NVIDIA pr-test/pr-test-extra: not run (rebase-gated, incomplete).

AMD Executed Failures

Job	Test File	Test Function	Error	Related?	Why
stage-a-1gpu-small	test/registered/attention/test_wave_attention_kernels.py	(file exit 1)	exit code 1	🟢	Wave-attention kernel; untouched by PR
stage-b-1gpu-small (3)	test/registered/lora/test_multi_lora_backend.py	(file exit 1)	Health-check/detokenizer timeout	🟢	LoRA on a base model; PR disables merge when LoRA active, not this path
stage-b-1gpu-small (11)	test/registered/attention/test_deterministic.py	(file exit 1)	exit code 1	🟢	Deterministic attention; unrelated
stage-c-large-8gpu (0-3)	test/registered/amd/test_moriep_small.py	TestEPLBMoriStat.setUpClass	scheduler died exit -3	🟢	MoRI EP / MoE; unrelated (verified shard 0; shards 1-3 same suite)
stage-c-mi35x (0)	test/registered/amd/perf/mi35x/test_qwen35_fp8_ar_fusion_mi35x.py	setUpClass	Server exited code -9	🟢	Qwen3.5-397B AR-fusion; -9 OOM on shared runner. Exonerated: shard-1 fusion test loaded the same default Qwen3.5 path and passed
stage-c-dsv4-flash-fp8-mi35x	test/registered/amd/test_deepseek_v4_flash_fp8.py	setUpClass	Server exited code -9	🟢	DeepSeek-V4 + hicache; unrelated, same -9 OOM pattern
stage-b-disagg-mi35x	test/registered/amd/disaggregation/test_nixl_transfer_engine_e2e.py	TestNixlTransferEngineAccuracy.setUpClass	Server exited code -9	🟢	NIXL disagg; unrelated, same -9 OOM pattern

Other Executed Failures

Job	Test File	Test Function	Error	Related?	Why
base-b-test-4-npu-a3	test/registered/npu/basic_function/dp_attn/test_npu_dp_attention.py	(file exit 1)	exit code 1	🟢	NPU DP-attention; PR is AMD-only
multimodal-gen-test-4-npu-a3	N/A	N/A	NPU job failure	🟢	NPU backend; unrelated
stage-a-unit-test-mlx	test/registered/unit/hardware_backend/mlx/test_scheduler_mixin.py	(file exit 1)	exit code 1	🟢	MLX scheduler mixin; unrelated
build-test (xeon-spr...)	test/registered/unit/server_args/test_server_args.py	(file exit 1)	ImportError: FlashinferA2ADispatchType from moe.utils	🟢	Pre-existing import breakage in moe/utils.py; PR doesn't touch it

(Cascade *-finish jobs and the 2 cancelled CPU build-test shards are collapsed into the above roots, not counted separately.)

Details / what to do before merge

• Rebase (required for any signal). Rebase onto latest main (base 3700c4ee) and re-push. Until then pr-test/pr-test-extra will keep hard-failing at check-changes, and the NVIDIA base-b kernel test for test_gdn_decode_fused_proj_conv.py (the +67-line addition) stays unverified. CI-fix exemptions aside, this gate applies to all PRs.
• AMD failures need no PR action — all are unrelated tests, and the MI35x -9 crashes are a shared-runner OOM cluster (multiple different large models crash the same way). Consider a re-run to confirm they're transient; they don't gate this PR's correctness.
• Optional: add the run-ci-extra label if you want the AMD extra suite; note the high priority/bypass-fastfail labels are already set, so a rebase + re-run should produce full-signal AMD + NVIDIA pipelines.

---

Generated by amd-bot using Claude Code CLI