Conversation
Load 3-byte W13 metadata cooperatively without changing dequantization or reduction order. Keep scalar W2 and the 4-byte path unchanged; document the component-only validation scope. Assisted-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose and stack
Layer 3/3, based on
perf/sm70-awq-qpn-runtime-20260905. This isolates thecooperative 3-byte W13 metadata reader from the native operator (layer1)
and default-off runtime admission (layer2).
Review order: #8 (prerequisite) -> #10 (kernel) -> #12 (runtime) -> #11 (3B loads).
distribute its 3-byte records with shuffles.
Bias reconstruction, MMA/reduction order, and FP16 boundaries are unchanged.
remains opt-in; no new flag or automatic layout switch is introduced.
This is not a separate general QPN framework or a duplicate of fork #8,
upstream 1CatAI#477, 1CatAI#349, or 1CatAI#509. Open SM70/AWQ/QPN PR checks found no competing
AWQ compact-metadata reader. Layer1 deliberately retains the scalar reader
so this change can be reviewed and reverted independently.
Test plan and results
The extracted CUDA source is byte-identical to the already GPU-validated
full native source, SHA-256:
55010c76911ba3e6ac6ed4017a6e624dfc3db2d1000221a28df6501d4c9481b5.The repository portable GPU test also matches the archived test exactly.
Changed-file pre-commit hooks and
git diff --check: passed.Both scalar and cooperative builds were tested with:
Per build: 2 portable GPU tests, 256 real-weight/dynamic-graph exact
comparisons to the same prototype, 72 FP64 stage checks with zero retained
rounding-bound violations, and 48 invalid argument rejections. Both
metadata layouts and all four logical TP partitions were covered. The
tiny-subnormal relative-only smoke exception is retained in the layer1 report;
scalar/cooperative/prototype outputs were exact for that case too.
A single-layer graph probe measured 3-byte scalar21.5280us versus
cooperative20.6160us (about4.2% lower). This is a component observation, not
an end-to-end speedup or statistical throughput claim. Full-model speed and
quality evidence in layer2 uses 4-byte metadata, so it must not be cited
as model-level validation of this 3-byte path.
Evidence:
qwen38-c1-qpn-awq-20260905/layer-gpu-{scalar,full}-r1.Full native core SHA-256:
e5083c1f26392873460c4a798244f454c6adef4acb66cc7d6573d4da8f8916e1.GPU jobs exited0/OOMfalse; logs/inspect were retained and GPU leases released.
This extraction did not introduce new arithmetic or collect new GPU timings.
AI assistance: OpenAI Codex assisted with implementation, analysis, tests and
documentation. Leon directed scope and requested submission to this fork.
This is a fork review draft, with final human review pending. No upstream
submission, production deployment or broad quality acceptance is claimed.