From d207fb901458717ec9ced65dfbbba3a0beed6be7 Mon Sep 17 00:00:00 2001 From: Yifan Qiao Date: Fri, 4 Sep 2026 02:04:26 +0000 Subject: [PATCH] [Bugfix] GLM-5.3-Flash: pass varlen indices to the kpool paged MQA logits On SM100 the indexer metadata builder schedules DeepGEMM's paged MQA logits in varlen mode (per-token rows tagged with request indices), but the kpool indexer launched the kernel without `indices`, so the non-varlen kernel ran on a varlen schedule. The varlen scheduler pairs adjacent rows of a request into one atom charged with the later row's context length; the non-varlen walk treats every row as its own request and stops only on an exact (q_atom, kv_split) end match. Once a pair straddles a 256-state split boundary the walk steps past an end marker it can never hit, the TMA warp stops prefetching Q and the math warps wait forever on the Q-stage barrier: 100% GPU utilization with idle host threads until the engine times out. Single-token decode has one row per request, so both schedules coincide and the bug never shows without speculative decoding. Co-Authored-By: Claude Fable 5.1 Signed-off-by: Yifan Qiao --- vllm/model_executor/layers/sparse_attn_indexer_kpool.py | 1 + 1 file changed, 1 insertion(+) diff --git a/vllm/model_executor/layers/sparse_attn_indexer_kpool.py b/vllm/model_executor/layers/sparse_attn_indexer_kpool.py index ba7a5adff160..b01e4c4adfed 100644 --- a/vllm/model_executor/layers/sparse_attn_indexer_kpool.py +++ b/vllm/model_executor/layers/sparse_attn_indexer_kpool.py @@ -792,6 +792,7 @@ def sparse_attn_indexer_kpool( decode_metadata.schedule_metadata, max_model_len=max_model_len, clean_logits=False, + indices=decode_metadata.indices, ) num_rows = logits.shape[0] # kpool: logits are pool-granular -> select topk_tokens//kpool pools,