[DSV4] Split mixed-chunk attention: route decode tokens to the fp8 paged MLA kernel

[yhyang201]

Motivation

With --enable-mixed-chunk, every step in the high-concurrency steady state is a mixed batch (prefill chunk + one decode token per running request), and all tokens go through the bf16 sparse-prefill workspace path. This forces two _dequantize_k_cache_paged calls per layer covering every running request's SWA window (~16.9 ms/step at conc 2048 on 8xB300), and the fp8-native decode kernel is never used in steady state.

Modification

mix_with_running appends decode requests after prefill requests (one token each), so decode tokens are the contiguous tail of the batch. Split at that boundary:

• decode tail -> fp8-native flash_mla_with_kvcache (row-sliced topk indices, no dequant);
• prefill head -> existing bf16 sparse path, with SparsePrefillChunkCache built over the prefill requests only, shrinking the dequant workspace.

Controlled by SGLANG_OPT_MIXED_SPLIT_DECODE_ATTN (default on; set to 0 to fall back to the previous behavior).

Results

8xB300, DeepSeek-V4-Pro FP4, 8k1k, conc 2048 (megamoe + dp-attention + mixed-chunk), radix cache enabled with forced misses, num_prompts = 6x conc, warmup = 2x conc. Back-to-back A/B on the same machine with byte-identical launch commands; the only variable is this commit (feature default-on for the comparison):

	Without this commit	With this commit	Delta
Output throughput (tok/s)	8983.25	9828.27	+9.4%
Median TPOT (ms)	196.25	177.20	-9.7%
P99 TPOT (ms)	196.89	178.37	-9.4%
Median TTFT (ms)	26925	24517	-8.9%
Successful requests	12288/12288	12288/12288	—

GSM8K under concurrency (mixed steps active): 0.970 with the flag vs 0.980 without, within noise (n=200).

TODO (draft)

• Done: num_mixed_decode_tokens is recorded by mix_with_running and plumbed through ForwardBatch. A temporary cross-check assert against a tail-scan of extend_seq_lens ran clean over 16k requests (~4.7k mixed steps) and has been removed.
• Full GSM8K (1319) + a long-context eval; conc 1024 regression check.

---

CI States

Latest PR Test (Base): ❌ Run #31791518956
Latest PR Test (Extra): ❌ Run #31791518814

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!