[Kernel][SM70] Accelerate exact Qwen3.8 decode

[yangzhuxinyzx]

Purpose

Raise Qwen3.8-Flash-Next-NVFP4 TP4/V100 no-MTP pure decode from the matched
checkpoint-native baseline to 80 tokens/s without online QPN8, a top1-only
LM-head shortcut, or a task-quality regression.

Base SHA: 62ad1e02693f4c857f3b7547cef1860ee54e8053
(onecat/main). The branch includes the Qwen3.8 correctness foundation from
#408. PR #398 remains the separate MTP4 owner.

Scope and contract

• Model: /data/models/RadixArk/Qwen3.8-Flash-Next-NVFP4.
• Four V100-SXM2-32GB GPUs, TP4/PP1, V2 runner.
• ModelOpt checkpoint NVFP4 experts, FP16 activation/KV, online QPN8 off.
• Flash-V100/FlashQLA, full CUDA Graph, prefix caching, aligned Mamba state.
• Max length 262,144, max batched tokens 2,048, max sequences 1.
• Speed gate: input 8,192, output 512, greedy, ignore_eos, five
  prefix-cache-reset repetitions; pure decode excludes TTFT/prefill.
• Quality gate: frozen GSM8K indices 8-23, xhigh chat template,
  temperature/top-p/top-k 1.0/0.95/20, seed 20260828, max output 4,096,
  natural EOS.

This is not the 2026-08-24 input-1,024/output-256 E4M3/QPN8 contract.

Implementation

• Add an exact-topology, default-off SM70 checkpoint-FP16 row-GEMV route for
  eight audited Qwen3.8 projection roles.
• Fuse the exact batch-one HyperConnection projection/mix path.
• Fuse the exact non-interleaved GDN QKVZ and b/a projections and write
  qkv/z/b/a outputs directly.
• Replace QSA's eight-pass composite-key radix selection with four score
  passes plus exact increasing-index pivot-tie compaction.
• Keep unsupported shapes, prefill, other dtypes, online QPN8, and
  speculative decoding on the existing fallback.
• Remove unused fused-W13 and shared-expert prototypes from the retained
  source.

The opt-in controls are:

• VLLM_SM70_QWEN38_FP16_GEMV=1
• VLLM_SM70_QWEN38_FUSED_HC_FP16=1
• VLLM_SM70_QWEN38_FUSED_GDN_INPUT_FP16=1

Test result

Matched pure decode

Route	Five steady samples (tok/s)	Mean	Mean TPOT
control	65.872, 65.882, 65.856, 65.855, 65.853	65.864	15.183 ms
candidate	80.367, 80.451, 80.463, 81.560, 80.822	80.732	12.387 ms

The candidate improves throughput by 22.57% and reduces TPOT by 18.41%.
Every repetition exceeds 80 tok/s; population standard deviation is
0.442 tok/s, and all five candidate token arrays are identical. Warm 8K
prefill median is 2,766.4 ms versus 2,777.6 ms for control.

Quality

• GSM8K: 15/16 raw, 15/16 strict.
• Natural stopping: 16/16; closed thinking: 16/16.
• Length caps: 0; structurally invalid outputs: 0.
• Historical same-prompt MTP4 reference: 15/16 raw, 14/16 strict.
• The sole candidate miss is the same reference miss (item 12, predicted 12
  versus expected 13).
• Repository repetition/character health: 16/16 pass, zero replacement
  characters, maximum same-token run 3, maximum repeated 50-character window
  count 6 (failure threshold 40).
• The natural-output quality run reports weighted pure decode at
  80.935 tok/s.

The optimized FP16 reductions are deterministic but are not claimed
token-for-token identical to the control after all numerical decision
boundaries. Acceptance is based on real-weight operator bounds plus the
frozen natural-output task and health gates; the route remains default-off.

Focused tests

• 25 passed:
  tests/models/qwen4_exp/test_qsa_ops.py and
  tests/models/qwen4_exp/test_sm70_fp16_gemv.py.
• QSA GPU exactness/prefill-batch/CUDA-Graph replay: 3 passed on the retained
  QSA source.
• FP16 GDN operator screen: QKV/z bitwise 256/256; b/a bitwise 253/256 and
  254/256, worst absolute differences 0.00024414 and 0.00003052.
• Ruff format/check, shell syntax, and git diff --check: pass.

Evidence

Full contracts, trace tables, operator results, rejected paths, quality
details, and local artifact paths are recorded in
docs/design/sm70_qwen38_nvfp4_decode.md and
docs/design/sm70_v100_migration_control.md.

AI assistance was used for profiling, implementation, tests, and
documentation. The source changes and reported results were reviewed locally.