docs: add W4A16-experts W8A16-dense static feasibility gate

[seanphan]

Static/toolchain feasibility gate for mixed W4A16-experts + W8A16-dense quantization, per issue #66. Repo-only; no GPU, download, or quantization run.\n\n- New attempt note: attempts/w4a16-experts-w8a16-dense-vllm/README.md (status: static-fit-only)\n- Confirms per-target mixed-scheme support is native to the vLLM compressed-tensors loader (config_groups dispatch), W4/W8 both route through WNA16, MoE path is wired via the merged PR 48918.\n- Names the true blockers: Glm5Next only exists via open/blocked vLLM PR 53906 (plus open bugs #54317/#54458), and no mixed checkpoint exists publicly; producing one needs an explicit resource lease.\n- Estimated weight footprint ~171.2 GB (~159.4 GiB), ~9% over the measured UD-IQ4_XS baseline, fits a 4x64 GiB node with ~85 GB headroom for KV/context.\n\nGates 2-5 (recipe, TP4 boot smoke, quality spot-check, full ladder) are written as pending-resource and stay closed until issue #66 records a lease.\n\nTest plan:\n- rg sanitization scan clean (no private hosts/IPs/paths)\n- git diff vs origin/main touches only the new attempt note\n- All size arithmetic shown inline and labeled inferred\n- No executable code changed; bench/ untouched

[seanphan]

PR-Agent gate blocked at exact head bc2bb6e8aeb20e92dfe8a8602485fa262b606355. No PR-Agent run or exact-head verdict exists. The repository default branch main does not contain .github/workflows/pr-review.yml, so an issue-comment /review cannot dispatch even though the Actions allowlist already includes PixelML/pr-agent/*. Blocker: onboard/merge the SHA-pinned caller workflow on main, then post /review again against the still-current head.

[seanphan]

CHANGES_REQUESTED at exact head bc2bb6e8aeb20e92dfe8a8602485fa262b606355.

The attempt is correctly placed and I found no sanitization issue. The following decision-critical claims need correction before approval:

1. [blocking: inferred/untested size] Recompute the artifact size from the complete pinned checkpoint inventory. The official checkpoint at 04c4e9e95c5da8862dced7e5056455116f83a7e0 contains 321,323,031,390 parameters, while this note counts 313,327,015,936, leaving 7,996,015,454 parameters unaccounted for. The omitted set includes MTP/vision and architecture-specific tensors, so 171.2 GB is not a complete artifact estimate. The 4 × 4096 × 8192 × 45 attention shortcut also does not represent the Glm5Next hybrid KDA/NoPE sparse-MLA projections in the pinned config. Recompute from the pinned safetensors index/config and state which tensors remain FP8/BF16 or are excluded.

2. [blocking: inferred size] Fully specify the compressed-tensors schemes before claiming exact packing. “Group-wise” does not define group size, symmetry, scale dtype, zero-point dtype, or target precedence. Applying one 1.0625 multiplier to both W4 and W8 is not generally valid. Pin a concrete config, show byte arithmetic per scheme including scales/zero-points/padding, and label the result inferred/untested until an emitted checkpoint is measured.

3. [blocking: unsupported runtime/topology] PR #53906 alone is not a serving path for the stated SM80 node. At the cited upstream head it implements the official GLM path for SM90/SM100/SM120, not SM80; PP is explicitly gated off, so PP4 is unsupported there. TP4 is only plausible/untested. On four PCIe GPUs without P2P, vLLM custom all-reduce is not supported for this topology and must be disabled; host-routed TP collectives are a material performance risk. Replace “no additional serving-side patch is required beyond #53906” with the actual blocker, separate any community backport from upstream vLLM, and label TP4 untested / PP4 unsupported. Also change “~85 GB headroom” to aggregate unallocated VRAM before runtime state, rank imbalance, quant buffers/repacking, graphs, NCCL, model-state omissions, and KV cache.

4. [blocking: inferred storage/evidence] The ~340 GB scratch requirement is not supported. The pinned official FP8 safetensors alone are 328,337,455,672 bytes; source plus a ~171–190 GB output exceeds 500 GB before temporary/offload data. Define whether “scratch” excludes separately stored source/output and give a toolchain-specific peak-disk/host-RAM estimate. Pin all model/runtime/quant source links to revisions; link the vLLM issues directly; and use the repository evidence labels consistently for every external, inferred, and untested claim.

Primary references: official pinned model, official pinned config, vLLM GLM support PR, compressed-tensors MoE integration, runtime issue #54317, and runtime issue #54458.

GitHub does not permit the PR author to submit a formal REQUEST_CHANGES review, so this COMMENT review carries the explicit CHANGES_REQUESTED verdict.

[seanphan]

Corrected per exact-head review 5063926089 (all 4 blockers):

• Replaced all byte totals: official pinned fp8 checkpoint = 321,342,220,638 params / 328,337,455,672 B = 305.8 GiB (62 shards, 76,108 tensors, rev 04c4e9e).
• Planned mixed-scheme total corrected to 170,526,899,714 B = 170.5 GB = 158.8 GiB = 1.087x the GGUF baseline (156,822,111,075 B); per-class provenance in the note.
• Runtime limits corrected: vLLM PR 53906 open/merge-blocked, SM90/100/120 only; PP gated off upstream (PP4 unsupported); TP4 plausible/untested; custom all-reduce unsupported on 4x PCIe without P2P and must be disabled. Storage: source 305.8 GiB + output ~170.5 GB + toolchain temps; per-tool peak deferred to gate 2 (prior ~340 GB figure retracted).
• Measured / inferred / untested labels applied; no serving or GPU claim added.

Fix head: f331479 (supersedes bc2bb6e via merge commit, no force-push). Requesting fresh independent exact-head review at this head.

[seanphan]

CHANGES_REQUESTED at exact head f331479572c27487f498a39ba3cc49f30d23d641.

The branch now contains merged PR #7 main as an ancestor; the exact-head diff against current main is only the new attempt note. git diff --check, commit metadata, large-file/symlink checks, and targeted publication-boundary scans pass. Four decision-critical blockers remain:

1. [blocking: arithmetic/inventory] The corrected artifact estimate still quantizes source scale tensors and the displayed formula does not equal its result. The pinned Hugging Face metadata reports 321,323,031,390 model parameters; the note's 321,342,220,638 “parameters” adds 19,189,248 auxiliary quantization-scale elements. At lines 55-56, 311,653,564,416 × 0.5 + (311,653,564,416 / 128) × 2 is 160,696,369,152 B, not 160,706,177,280 B. The stated result is exactly what results from first adding the existing 19,021,824 weight_scale_inv elements and then W4-packing them, even though those source FP8 scales must be replaced—not treated as model weights in the emitted scheme. Likewise, the pinned index metadata.total_size is 328,326,771,576 tensor-payload bytes; if 328,337,455,672 is the 62 LFS blob sum including safetensors headers, label that distinction. Recompute output bytes from the emitted tensor classes, separately reporting model parameters, source auxiliary scales, new scales, padding, and shard/header overhead. Primary receipt: pinned safetensors metadata and pinned index.

2. [blocking: the scheme/tensor map is still not frozen and conflicts with the owner contract] Lines 12-21 define W8 for “all other Linear” while preserving norms/KDA/non-linear state, but lines 57-60 apply W8 + scale overhead to every remaining tensor element. That bucket includes embeddings, norms, KDA/state tensors and source auxiliary scales, so it cannot implement the stated target set. The tracking contract also preserves embeddings/output ties, biases, routers and unsupported state, and requires shared-expert treatment to be decided from the frozen tensor map. The note commits neither a concrete config_groups manifest nor an exhaustive included/excluded tensor inventory. Also, the source FP8 [16,32] scale layout is 128×128 block scaling, while the estimate assumes one output FP16 scale per 128 weights; it does not “verify” the planned WNA16 grouping. Commit the exact config and machine-checkable per-tensor classification/counts before claiming gate 1 complete.

3. [blocking: runtime/quant claims remain mutable rather than pinned] PR #53906, PR #48918, llm-compressor #2940, “vLLM main compressed_tensors.py,” and the community backport are all linked or named without immutable revisions. #53906 is already at f221389f16e02712b0e8bd82a485017a24a18453, different from the head used in the original research. Pin commit URLs for every implementation claim, link/license the chosen community backport, and record status as-of the review date. Reconcile gate 3's TP4 plan with the current #66 decision record, which now prefers a separately pinned community PP4 lane after import validation.

4. [blocking: PR body and tracking link are stale] The public PR body still says 171.2 GB / 159.4 GiB and “~85 GB headroom for KV/context,” contradicting the note's 170.5 GB / 158.8 GiB / 97.2 GiB not-serving-headroom wording; it also contains literal \n escapes. Its unqualified “issue #66” points at a nonexistent issue in this repository rather than the actual owner ticket. Update the body after the artifact math/tensor contract is corrected.

This review is a COMMENT because GitHub does not allow the PR author/account to submit a formal request-changes review; it carries the explicit CHANGES_REQUESTED verdict for this SHA.

[seanphan]

VERDICT: CHANGES_REQUESTED

Independent exact-head review of f331479572c27487f498a39ba3cc49f30d23d641 (merge base 2bf65afc = current main tip; true landing diff = attempts/w4a16-experts-w8a16-dense-vllm/README.md only). Formal review API is unavailable for this identity, so this comment carries the verdict, same convention as the two prior reviews.

1. [blocking — internal arithmetic, README.md:55-61] The W4 line's displayed result does not equal its own formula: 311,653,564,416 × 0.5 + (311,653,564,416 / 128) × 2 = 160,696,369,152 B, not 160,706,177,280 B. The 9,808,128 B delta is exactly the source weight_scale_inv (19,021,824 F32 elements) W4-packed as if it were model weights; the planned total at line 61 inherits the same +9,808,128 B (formula floor: 170,517,091,586 B). Separately, "321,342,220,638 parameters" (line 28) overstates the HF safetensors parameter total of 321,323,031,390 by 19,189,248 auxiliary scale elements — relabel it and report source scales, new scales, and header/shard overhead separately.

2. [blocking — scheme map contradicts itself, README.md:12-21 vs 57-58] The W8 numerator 9,669,634,398 still contains tensors the same note declares excluded from quantized math: hc_* KDA coefficients (35,391,870) and norms (389,120) are non-Linear and "stay BF16", yet sit inside "everything else W8". No concrete config_groups manifest or per-tensor included/excluded inventory is committed, and the source FP8 [16,32] layout (128×128 blocks) does not verify the planned per-128-output FP16 WNA16 grouping — the two schemes differ in scale shape and count.

3. [blocking — unpinned mutable references, README.md:111,130] vLLM PR 53906 is open/unmerged and its head has moved again (now 1f0369b05d53486ea9360623bacf257455946bbb). llm-compressor PR 2940 is still open, cited as "precedent" with no revision or state label. "vLLM main compressed_tensors.py, inspected 2026-08-31" is a floating reference — get_scheme_dict and _add_fused_moe_to_target_scheme_map do exist on main today, but the claim needs a commit SHA. The wtdcode community backport remains unnamed, unpinned, and unlicensed.

4. [blocking — stale, self-contradicting PR body] The body still claims 171.2 GB / 159.4 GiB / "~85 GB headroom", contradicting the note's corrected 170.5 GB / 158.8 GiB / 97.2 GiB explicitly-NOT-serving-headroom. It still says "git diff vs origin/main touches only the new attempt note" under a 57-file changed-file listing (stale open-date base 0eab34e), and contains literal \n escapes. The unqualified "issue #66" returns 404 in this repository — link the actual tracking ticket.

5. Verified clean at this head: the true main...f331479 diff is exactly the one new note (merge base equals the main tip; the 57-file Files tab is a stale-base display artifact, not landing scope); measured/inferred/untested labeling is disciplined; ttft_s: 0.0 is disclosed as a harness artifact in results/phase63/README.md and never presented as a real measurement; results/summary.csv (0.7833, 17.73 tok/s) reconciles with quality-final2.jsonl (21/26) and the 5-rep median in speed-c1.jsonl; the 156,822,111,075 B / 146.05 GiB baseline matches the GGUF attempt's measured blob sum; the sanitization regex sweep over all 57 files found only 127.0.0.1 loopback endpoints — no private IPs, tailnet addresses, hostnames, VMIDs, GPU UUIDs, or secrets.

6. Scope: acceptable as one docs PR only because the true landing diff is the single note — relabel the body to describe that scope (and drop the 57-file impression); no split required. Meta: per the issue comment of 07:07Z, the pr-agent caller workflow is still absent from main, so no PR-Agent gate has run at this head either.

Re-review at the next head should confirm: W4/W8 formulas equal their displayed results, scales reported separately from model parameters, a frozen config_groups manifest with per-tensor counts, commit-pinned runtime/quant references, and a body matching the note.