perf(glm5next): use direct single-token KDA decode

[jackzampolin]

Resulting behavior

Ordinary GLM-5.3 decode dispatches directly to the B12X single-token KDA specialization when the installed B12X exposes it. The direct path forwards caller-owned tensors and scheduler-owned state indices without copying through the generic packed-decode staging buffers.

Speculative, mixed, and variable-width decode remain on run_kda(), including its device-side metadata and transaction validation. Older B12X installs keep the existing path through feature detection.

Dependency

This is the vLLM integration counterpart to local-inference-lab/b12x#249. It is safe to merge before that PR because the dispatch is guarded by hasattr(api, "run_kda_single_token").

Validation

• Ruff check and format pass for both changed files.
• git diff --check passes.
• tests/models/test_glm5next_model.py -k b12x_kda: 3 passed, 28 deselected.
• B12X [GG] Support heterogeneous per-layer routed-expert widths (deepseek_v2 + EXL3) #249 GPU correctness suite: 16 passed, including numerical reference checks, CUDA-graph replay, rejected-draft rollback, null-state safety, and GLM TP8 geometry.
• Added identity assertions proving ordinary decode forwards the original tensors and does not invoke packed KDA staging.
• Source-locked TP4/DCP4/MTP5 GLM-5.3-Flash-NVFP4 canary completed both CUDA-graph capture stages and remained healthy with zero container restarts.
• Matched synthetic C1 sustained decode improved from 100.8 to 122.1 tok/s (+21%); prefill remained approximately 8.2k tok/s, as expected for this decode-only change.
• Deterministic hot-prefix Estonia completed correctly at 201.9 tok/s with 72.8% draft-token acceptance, 4.64 emitted tokens per target step, and 0.82s TTFT.
• Prefix caching was independently verified with identical 126k-token prompts: 14.99s cold versus 0.76s hot and 122,112 cache-hit tokens.

The serving measurements are single-run canary evidence, not a general performance guarantee. The separate B12X sparse-MLA prefill regression remains outside this PR.

Duplicate check

Searched current open vLLM PRs for GLM, B12X, KDA, and decode work. No existing vLLM PR integrates the B12X #249 single-token API.

Assisted by OpenAI Codex. The operator reviewed the runtime scope; final human review remains required before merge.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[lukealonso]

duplicates #504