fix(qwen3.5): keep CUDA dual-stream overlap (regressed by #25885)

[YAMY1234]

Motivation

#25885 ("[AMD] Support alt stream for Qwen3.5 on AMD platform") gated three existing CUDA dual-stream (alt_stream) overlaps in qwen3_5.py behind AMD-only conditions: _gdn_use_alt_stream and _qknorm_use_alt_stream both require _is_hip, and the MLP alt_stream is forced to None on the (default) shared-experts-fusion path. These overlaps are therefore silently disabled on all CUDA GPUs (the original code enabled them whenever _is_cuda, with no arch gating), with no env var to re-enable them — regressing Qwen3.5 decode throughput. Hopper (H200) is simply where it was first observed in CI; Blackwell/Ampere are affected too.

git bisect on the Qwen3.5-FP8 conc4 / isl8192 / osl1024 workload (H200, TP8/EP8) identifies 7f919ed (#25885) as the first bad commit.

Modifications

Preserve the pre-#25885 behavior on CUDA while keeping the new AMD env gating intact:

• Qwen3_5GatedDeltaNet._forward_input_proj: and (_is_cuda or _gdn_use_alt_stream)
• Qwen3_5AttentionDecoderLayer._apply_qk_norm: and (_is_cuda or _qknorm_use_alt_stream)
• MLP / shared-expert alt_stream: (alt_stream if (_is_cuda or _disable_shared_experts_fusion()) else None)

Accuracy Tests

GSM8K via sglang.test.run_eval --eval-name gsm8k --thinking-mode qwen-3 --max-tokens 16384 (chat API + thinking, 1319 examples), Qwen3.5-397B-A17B-FP8 on H200:

	GSM8K
main (unfixed)	0.979
this PR (fix)	0.977

Within run-to-run variance — the change only re-enables CUDA stream overlap and does not alter the forward computation.

Speed Tests and Profiling

Qwen3.5-397B-A17B-FP8, 8×H200, TP8/EP8, flashinfer attention, random isl=8192 osl=1024, concurrency=4, --quantization fp8 --kv-cache-dtype fp8_e4m3:

	Output tok/s	Mean TPOT (ms)
Before (current main)	377	9.97
After (this PR)	434	8.60

~15% decode-throughput recovery, matching the pre-#25885 baseline (verified by checking out the parent commit in the same image).

---

CI States

Latest PR Test (Base): ✅ Run #27525425492
Latest PR Test (Extra): ❌ Run #27525425339

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[YAMY1234]

/tag-and-rerun-ci

[buaasun]

I tested this PR on another Qwen3.5-MoE-family workload with the official SGLang benchmark, and the result looks positive.

Environment / setup:

• GPU: NVIDIA H20
• Model: Qwen3.6-35B-A3B
• Benchmark: python -m sglang.bench_serving
• Dataset: random-ids
• Input length: 1000
• Output length: 512 / 2048 / 5000
• --random-range-ratio 1.0
• --num-prompts 10
• --max-concurrency 1
• --request-rate inf
• Serving flags included FA3, disabled radix cache, CUDA graph TC compiler with inductor, context/max-total-tokens 8192.

Results:

Output len	Clean main output TPS	This PR output TPS	Speedup	Clean TPOT	This PR TPOT
512	202.39 tok/s	230.31 tok/s	+13.80%	4.78 ms	4.16 ms
2048	207.34 tok/s	237.57 tok/s	+14.58%	4.78 ms	4.17 ms
5000	207.74 tok/s	238.33 tok/s	+14.72%	4.80 ms	4.18 ms

All cases used 10 successful requests and generated the configured output token count.

I also compared against a narrower shared-expert-only dual-stream variant; this PR was another ~3.7-4.2% faster in output TPS on the same benchmark, likely from additionally restoring the GDN input-proj and QK-norm alt-stream overlap.

One small note: applying the patch locally reported a trailing blank/whitespace warning, so the lint failure may be due to a simple formatting issue. Adding unit coverage for the CUDA alt-stream routing would also make this easier to review, but the performance direction looks correct from this independent run.

[ispobock]

/rerun-failed-ci

[ispobock]

can you fold _is_cuda to the flag?
for example:

_gdn_use_alt_stream = _is_cuda or (
    get_bool_env_var("SGLANG_GDN_QKVZ_BA_ALT_STREAM", "False") and _hip_use_alt_stream
)
_qknorm_use_alt_stream = _is_cuda or (
    get_bool_env_var("SGLANG_QK_NORM_ALT_STREAM", "False") and _hip_use_alt_stream
)

[YAMY1234]

Thanks! Revised~