Skip to content

[Perf][SM70] Coalesce AWQ QPN W13 compact metadata loads - #11

Draft
Leonccaa wants to merge 1 commit into
perf/sm70-awq-qpn-runtime-20260905from
perf/sm70-awq-qpn-3b-loads-20260905
Draft

Leonccaa wants to merge 1 commit into
perf/sm70-awq-qpn-runtime-20260905from
perf/sm70-awq-qpn-3b-loads-20260905

Conversation

@Leonccaa

@Leonccaa Leonccaa commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Purpose and stack

Layer 3/3, based on perf/sm70-awq-qpn-runtime-20260905. This isolates the
cooperative 3-byte W13 metadata reader from the native operator (layer1)
and default-off runtime admission (layer2).

Review order: #8 (prerequisite) -> #10 (kernel) -> #12 (runtime) -> #11 (3B loads).

  • Read each 96-byte metadata tile with 24 aligned 32-bit loads per warp and
    distribute its 3-byte records with shuffles.
  • The final record at byte93 neither reads nor shuffles beyond the tile.
  • W2 keeps the scalar 3-byte reader. The 4-byte path is unchanged.
  • No metadata representation change, repack, added weight copy or buffer.
    Bias reconstruction, MMA/reduction order, and FP16 boundaries are unchanged.
  • Removing this commit restores the scalar 3-byte path. The overall runtime
    remains opt-in; no new flag or automatic layout switch is introduced.

This is not a separate general QPN framework or a duplicate of fork #8,
upstream 1CatAI#477, 1CatAI#349, or 1CatAI#509. Open SM70/AWQ/QPN PR checks found no competing
AWQ compact-metadata reader. Layer1 deliberately retains the scalar reader
so this change can be reviewed and reverted independently.

Test plan and results

The extracted CUDA source is byte-identical to the already GPU-validated
full native source, SHA-256:
55010c76911ba3e6ac6ed4017a6e624dfc3db2d1000221a28df6501d4c9481b5.
The repository portable GPU test also matches the archived test exactly.
Changed-file pre-commit hooks and git diff --check: passed.

Both scalar and cooperative builds were tested with:

/evidence/test-venv/bin/python -m pytest -q --confcutdir=/tests \
  /tests/test_sm70_awq_qpn_m1.py
/evidence/test-venv/bin/python /native-tools/probe_native.py \
  --manifest /output/native-manifest.json --output /output/probe
/evidence/test-venv/bin/python /native-tools/benchmark_layer_split.py \
  --output /output/split-timing.json

Per build: 2 portable GPU tests, 256 real-weight/dynamic-graph exact
comparisons
to the same prototype, 72 FP64 stage checks with zero retained
rounding-bound violations, and 48 invalid argument rejections. Both
metadata layouts and all four logical TP partitions were covered. The
tiny-subnormal relative-only smoke exception is retained in the layer1 report;
scalar/cooperative/prototype outputs were exact for that case too.

A single-layer graph probe measured 3-byte scalar21.5280us versus
cooperative20.6160us (about4.2% lower). This is a component observation, not
an end-to-end speedup or statistical throughput claim. Full-model speed and
quality evidence in layer2 uses 4-byte metadata, so it must not be cited
as model-level validation of this 3-byte path.

Evidence: qwen38-c1-qpn-awq-20260905/layer-gpu-{scalar,full}-r1.
Full native core SHA-256:
e5083c1f26392873460c4a798244f454c6adef4acb66cc7d6573d4da8f8916e1.
GPU jobs exited0/OOMfalse; logs/inspect were retained and GPU leases released.
This extraction did not introduce new arithmetic or collect new GPU timings.

AI assistance: OpenAI Codex assisted with implementation, analysis, tests and
documentation. Leon directed scope and requested submission to this fork.
This is a fork review draft, with final human review pending. No upstream
submission, production deployment or broad quality acceptance is claimed.

Load 3-byte W13 metadata cooperatively without changing dequantization or reduction order. Keep scalar W2 and the 4-byte path unchanged; document the component-only validation scope.

Assisted-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant