docs(docs): add ADR-0028, the expert-stack type table - #227
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the GGUF type-mapping documentation to address routed-expert stacks that cannot use k-quant types due to row-size constraints, by introducing a dedicated expert-stack type table and updating related references across the ADR index and glossary.
Changes:
- Add ADR-0028 to define an expert-stack-specific nominal-bits → GGUF-type mapping and specify pack-time behavior for nominal 3 and type-fallback warnings.
- Amend ADR-0012 to reference ADR-0028 and narrow the open question around toolchain-output persistence.
- Update the glossary and ADR index to include and reference ADR-0028.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| docs/reference/glossary.md | Updates the “Type mapping” glossary entry to reference the expert-stack carve-out and ADR-0028. |
| docs/adr/index.md | Adds ADR-0028 to the ADR index and notes ADR-0012 is amended by it. |
| docs/adr/0028-expert-stack-type-table.md | New ADR defining the expert-stack type table and pack-time halting behavior on type fallback. |
| docs/adr/0012-gguf-type-mapping.md | Amends ADR-0012 header and open-question text to incorporate ADR-0028. |
Suppressed comments (1)
docs/adr/0028-expert-stack-type-table.md:116
- This paragraph pins llama.cpp to
e9fa0781, but other ADRs reference the same checkout ase9fa078. Use a consistent short SHA across docs to avoid implying a different revision.
2026-07-07. The pinned checkout `e9fa0781` carries it (#159).
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+15
to
+16
| this after a manual `--tensor-type` override (`src/llama-quant.cpp:374` | ||
| and `:712`, checkout `e9fa0781`). A rejected override degrades to |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ADR-0012's table emits only k-quants below 8 bits, and every k-quant needs
rows divisible by 256. The 30B target's expert stacks carry rows of 2688 and
1856, so
llama-quantizerewrites every k-quant type on 93.0 % of theparameters — after the manual override, on a zero exit, with a warning pack
discards. The packed file stops being recipe-driven, which ADR-0012
decision 3 forbids, and the Destination's 2-bit tier has no table entry at
all. #189 carries the defect; the maintainer ruled it lands as a new ADR
with the fallback-warning capture folded in.
(8→Q8_0, 4→Q4_0, 2→Q2_0), the backend refuses nominal 3 on the empty
2.25–4.25 gap, and a type-fallback warning halts the pack
toolchain-output open question, and add the index row
Test:
pre-commit run --all-files(docs-only; docvet, doc-refs, andbanned-terms gates pass)
Resolves the question on #189 — the ticket closes manually with its
pointer comment, not by this PR, so the record link lands first (the
PR #184 lesson). #228 carries the build.
PR Review
Checklist
pre-commit run --all-filesand a push-stage run)!in title andBREAKING CHANGE:in body — n/aReview Focus
The decision 1 table's 4-bit row: Q4_0 over MXFP4, argued from
quantize_q4_0consuming the per-expert imatrix slice whilequantize_mxfp4ignores it. The swap condition is an open question inthe record. Also decision 3's halt-vs-record split against the ADR-0016
imatrix-miss scan.
Related
#189 (chart:discuss, chart #158), #228 (build ticket), #159 findings,
ADR-0012, ADR-0021 decision 4 (untouched — the table changes meaning,
not the 2-bit buying bar), #183 (owns the remaining Nemotron-H classes),
upstream llama.cpp #24448.