[II] Execute Brandon GLM-5.2 SQG W4A8

[voipmonitor]

Status

Implemented and qualified on SM120. Human review is required before merge.

Purpose

Serve brandonmusic/GLM-5.2-SQG-W4A8 revision
593dd0d2de6f79ce4e65303930c22c75e1359d44 through native B12X kernels.
The checkpoint uses Luke Alonso's SQG reconstruction primitive. Its independent
K3/K4/K6 allocation and BMM-Law calibration form Brandon's GLM profile, not the
canonical paired-rate QSRT format.

Resulting behavior

• Execute routed gate, up, and down matrices through a route-packed W4A8
  endpoint with independent K3/K4 expert partitions and direct E4M3 weights.
• Execute uniform SQG K6 dense matrices through a W6A16 endpoint. The SM120
  small-row kernel decodes the exact T12 codebook inline; larger row counts use
  the generic Trellis scheduler.
• Plan route, transform, quantization, output, and GEMM scratch storage before
  CUDA graph capture. Replay performs no allocation.
• Reject incompatible codebooks, paired payloads, bitrates, architectures, and
  unqualified launch schedules before execution.
• Preserve MCG, projection-mixed EXL3, and canonical QSRT paths.

Dependency and scope

The comparison base agent/pr197-base-glm52-sqg is a read-only integration of
the exact heads of B12X PRs #145, #146, #148, #149, and #150. It gives reviewers
a one-commit SQG diff while those prerequisites remain unmerged. Merge those
five PRs into master, retarget this PR to master, and delete the comparison
base branch.

The SQG implementation preserves PR #150's preallocated FC1 preparation output;
it does not restore the allocation-producing torch.stack form. This PR does
not duplicate PR #129, which serves canonical Fruit QSRT atoms, or PR #149,
which validates the W4A8 rate contract without adding the GLM route-packed
runtime.

Validation

• Source contract suite in the CUDA 13.3 runtime: 60 passed, 34 CUDA-only
  tests skipped.
• SM120 suite on physical GPU 4 of 192.168.0.69: 94 passed in 118.12 s.
• The GPU suite exhaustively validates all 65,536 K6 codewords, compares dense
  and routed outputs with independent references, checks scratch upper bounds,
  and replays both endpoints in CUDA graphs.
• TP4/DCP1/MTP0 serving on root-port RTX PRO 6000 Blackwell GPUs used full CUDA
  graphs: 31.3 tok/s C1 decode and 2,856 tok/s 8K prefill.
• Teacher-forced KLD against official BF16 logits: 0.08101 with FP8 KV and
  0.12375 with NVFP4 KV. The BF16-equivalent KV diagnostic measured
  0.07865.

AI assistance: OpenAI Codex was used for implementation, profiling, tests, and
PR drafting. A human maintainer must review and defend the change before merge.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 79276921-2b80-4fa3-bb67-5c0a43ebf802

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

Qualified release evidence

Status: qualified for brandonmusic/GLM-5.2-SQG-W4A8 revision 593dd0d2de6f79ce4e65303930c22c75e1359d44 in TP4/DCP1/MTP0 with native B12X SQG execution and NVFP4 DS-MLA KV.

The immutable release image contains this pull request's head and no private source patch:

voipmonitor/vllm:infernal-invocation-vllm0af7310-b12xec2f97d-fi1ac6942-cu133-torch213-20260814-r13
sha256:a9059b81458167ffd27df77c17e928af81650404eff3e7f90ad528a16c96b10b

Validation:

• 60 CPU/source tests and 94 physical-SM120 tests passed.
• The GPU suite covered all 65,536 K6 codewords, independent decode oracles, scratch bounds, and CUDA graph replay.
• All four rank-evidence records listed routed layers 3 through 77 as loaded and executed through full-w4a8; A16 fallback was disabled.
• FULL and PIECEWISE CUDA graphs captured rows 1 through 6.
• Two warmed 512-token CC1 runs reached 31.02 and 31.26 tok/s.
• Three uncached 8,192-token prefills reached 3,040.9, 3,025.3, and 3,043.7 tok/s.

Runbook: https://github.com/local-inference-lab/rtx6kpro/blob/master/models/glm5.2-infernal-invocation-r13-sqg.md

Merge contract: local-inference-lab/rtx6kpro#67

This pull request remains draft because its comparison base is the exact integration of B12X PRs #145, #146, #148, #149, and #150. Merge those prerequisites into master, retarget this pull request to master, and verify that the remaining semantic diff is limited to native GLM SQG operators and tests before merge.

[lukealonso]

Closing because this branch is incompatible with the repository master execution contract. It targets agent/pr197-base-glm52-sqg, depends on unmerged PRs #145, #146, #148, #149, and #150, and modifies native Trellis files that are absent from master, including b12x/gemm/trellis_linear/csrc/trellis_k6_small.cu. The patch also introduces model-specific dense and MoE kernel paths rather than a small addition to an existing shared kernel.

GLM-5.2 SQG support should be proposed from master without a standalone native Trellis module and validated for correctness, CUDA graph replay, allocation stability, and comparative performance on the resulting implementation.