[Kernel][WIP] Continue exact single-token HC latency optimization

[yangzhuxinyzx]

Purpose

Continue the single-request/no-MTP Qwen3.8 HC <=1.5 ms work after #481 merged.
Integration base: public/main 755baae1d0; integration merge 4b6c2daa1f.
This is separate from #504's batched HC scope. Do not merge/promote on the
basis of microbenchmark numbers: the 1.5 ms full-HC target is NOT achieved.

Implementation

• Decode-only HC norm weight prefetch, source 8f74e4b88b: only SM70, FP16,
  N=1, HC=4, H=2560. Other shapes/dtypes/architectures and prefill unchanged.
• Preserve original FP32 arithmetic/reduction order and FP16 boundaries.
  Only move the weight load; no public flag or extra weight copy.
• [Perf][SM70] Reduce Qwen3.8 no-MTP token latency #481 already contains the registered exact up/mix/gather fusion. It is a
  dependency, not a new implementation claim in this PR.

Test Plan and Result

• Registered full-HC paired microbenchmark: late 1.982525 -> early 1.944255 ms
  (1.93%); all intermediate/final FP16 bits match on 4 ranks x16 inputs and
  after timing. Up fusion fixed on both sides. No new communication here.
• Post-integration targeted CPU dispatch/owner tests: 35 passed.
• One full-model retry at frozen PRE-integration source e76a9c8ca3 succeeds:
  two thinking-enabled natural-EOS official-sampling checks pass (118/110
  outputs). 8K/513 untraced pure decode 93.433729 tok/s, 10.702773 ms/token;
  prefill 1.162319 s. This is one measured repeat, not an endpoint campaign.
• Same Nsight 2022.4.2.50 graph-node/rank-ordinal criterion, 29 middle tokens:
  old HC bucket 2.658072 -> 2.165077 ms; complete semantic HC
  2.701252 -> 2.208302 ms. Rank-max mean 2.237414, rank-average p50 2.188509.
  Outliers are retained. GPU service sums are not additive wall critical path.
• Exact workload: Qwen3.8-Flash-Next-NVFP4, 4x V100-SXM2-32GB GPUs0-3, TP4/PP1,
  V2, FP16 A/KV, checkpoint-native NVFP4, no online QPN8, no MTP, no prefix
  cache, max context262144, q8192, dual graphs, hybrid PLE; Torch2.10.0+cu128.
• Tests release all owned GPU/model/PLE processes. No task API left resident.

Evidence and Remaining Gates

Retained worktree:
/home/ymzx/桌面/1cat-vllm/worktrees/v100-qwen38-nomtp-token-trace-20260903-173451.
Artifacts .artifacts/hc_norm_prefetch/registered_result.json and
.artifacts/hc_trace_20260905_retry1/ include contract, logs, quality/result,
raw/converted trace and per-token tables. Hashes and rejected variants are
recorded in docs/design/sm70_v100_migration_control.md.

First full-model attempt failed BEFORE generation because the quality harness
passed tokenizer Mapping keys as IDs. CPU-validated correction precedes the
single successful retry; preserve the failed logs. Do not call it a model
quality failure or hide the startup.

The 256K INPUT quality boundary, repeated endpoint evidence, integration-head
GPU validation and <=1.5 ms acceptance remain pending. The new trace includes
earlier W13/W2 changes too; do not attribute all endpoint gains to HC. Next
bounded candidate is H8/512-thread fused-up scheduling, not the rejected
H8/256-thread geometry. No numerical contract change is allowed.

AI assistance: OpenAI Codex. Commits are DCO-signed; human review is required.

H8 scheduling screen completed

At 2c2ec3f38d, full-HC production 1.948767 ms, private H4 control 1.970251,
private H8/512 candidate 1.953417. Although geometry improves the private
control by 0.016835 ms, the candidate does not beat production and is NOT
ported/admitted. All 4 ranks x16 inputs, 512 actual auxiliary sum2 replays and
post-wrap generation195841 checks are bitwise. Both fused kernels 30 registers,
zero spills. Artifact .artifacts/hc_up_h8_threads/result.json. All test
processes exited and GPUs0-3 returned to 141/7/7/7 MiB. No further model start.

Current down counters and vector output scatter (4ae6a00)

The old NCU report measured 324 replicated down rows. A focused current
81-live-row/grid88/block128 profile instead finds 6.51% achieved occupancy,
216.88 GB/s, 91.84% no-eligible scheduler cycles and long-scoreboard 63.9%.
Clock-control none/cache-control all; cold NCU duration is not endpoint timing.
Existing trace collective-ordinal analysis shows 4.888424 us/call remain after
the last rank starts (5.620995 us total average service).

SASS identifies scalar scatter/routing overhead. Keep contiguous low-rank rows
packed as 16-byte output stores; separately scatter injection/padding, retain
unaligned scalar fallback. No arithmetic/protocol/IPC/epoch/up/norm change.
Paired full-HC test with old scalar and new vector in the SAME owner DSO:
1.971644 -> 1.899800 ms, saving 0.071844 ms (3.64%). All four ranks pass
raw-bit aligned/unaligned/guard tests, 16 changing full-HC inputs, 512 actual
auxiliary sum2 replays and post-timing bitwise checks. The source and independent
model-free public verifier are committed. The public verifier's additional
eight-alignment/reference GPU run is queued behind another task's reservation,
not yet claimed passed. This turn performed NO full-model start.

Evidence .artifacts/hc_down_ncu_20260905/ and
.artifacts/hc_down_vector_scatter/; result/binary/source hashes in worklog.
Do not subtract the micro gain directly from the previous 2.208302-ms model
trace. Full-model/256K/1.5-ms gates remain pending. An explicit down load-ahead
attempt was rejected offline because compiler rescheduling erased the intended
lookahead; no redundant GPU experiment was run for it.