This repository was archived by the owner on Oct 6, 2026. It is now read-only.
Repository navigation
[Research] Isolate optional stable W4A16 route packing - #437
Draft
lukealonso wants to merge 5 commits into
Draft
lukealonso wants to merge 5 commits into
lukealonso wants to merge 5 commits into
Conversation
Preserve source-coordinate expert extents, retained two-bit lookup tables and declared row capacities. Keep block-codec epilogues and pipeline tuning intact. Optional FP32 atomic prefill reduction does not guarantee bitwise summation order. Validation: Host eligibility checks pass for packed, ModelOpt, EXL3 and three block codecs. Syntax and whitespace checks pass. GPU validation of the canonical-base composition remains pending. Co-authored-by: Martin Vit <martin@voipmonitor.org> Co-authored-by: Codex <noreply@openai.com>
Normalize uniform EXL3 slots into TrellisSource and TrellisWeights. Preserve codewords, FP16 scale boundaries, global Hadamard coordinates and bounded CUDA staging. Route the compatibility loader through common preparation while retaining asymmetric-pair support. Native config callers keep their schema and projection-tiered path. CPU layout and adapter validation: 98 tests passed. GPU preparation, graph replay, performance and full-model fidelity remain unqualified in this environment. Assisted-by: OpenAI Codex
…packing Remove the full-codeword shared LUT and stable per-expert route ordering from the preparation interface change. Retain exact route capacities, prepared launch ownership and opt-in FP32 prefill reduction. Bump the MoE query schema for the reduced control set and scope the release fragment to PR 433. Validation: selected host suites report 181 passed and 28 failures; the same 28 missing-decode_config failures reproduce at 7fca2aa. Ten LUT-specific cases move with the LUT change. Changed Python sources parse and git diff --check passes. GPU execution and serving qualification were not run. Assisted-by: OpenAI Codex
Preserve ascending per-expert route ordering behind its frozen opt-in control, independently of common trellis preparation. Retain the original expert-by-route scan and its ordering tests. Document its work complexity and the absence of a deterministic output or measured performance contract. Validation: adapter, config, schema and variant-selection suites: 160 passed. Python source matches the extracted implementation except for the query schema version; syntax and whitespace checks pass. CUDA packing and graph replay were not run. Source-commit: 7fca2aa Assisted-by: OpenAI Codex
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
lukealonso
changed the base branch from
feat/trellis-common-preparation
to
master
September 27, 2026 15:51
This was referenced Sep 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Behavior
Add optional ascending token-major route order within each expert to prepared W4A16 execution.
B12X_W4A16_STABLE_ROUTE_PACK=1is frozen at declaration and applies at capacities of at least 4096 routes. Default packing remains unordered and atomic. Expert selection and router weights do not change.Status: research-only. Stable packing does not make floating-point MoE output deterministic. This draft requires a demonstrated ordering requirement or measured performance benefit before adoption.
Scope and alternatives
This PR targets
masterafter #433 merged and contains only the stable-packing implementation, frozen control, tests and documentation removed from that PR. The full-codeword LUT is absent. Runtime code is preserved from7fca2aa55d5eeb7116a4c09633632cc5f02d850d.The scan executes once per expert over the full route capacity, so work scales as experts multiplied by routes. #327 contains related stable-packing work, including a different segment-sort implementation on an older serving lineage. This extraction preserves #433's prepared-capacity implementation for separate review; those overlapping alternatives should not be merged together automatically.
Validation
Using
/home/luke/projects/b12x/.venv/bin/pythonagainst this isolated worktree:160 passed. Source comparison verifies exact extraction of Python behavior apart from query-schema version 16. Python parsing and
git diff --checkpass. These host tests do not validate the GPU packing kernel. CUDA route-pack tests, graph replay and performance were not run; they remain qualification gates.AI assistance: OpenAI Codex. Original contributor commits remain in the branch ancestry.
The merge of
masterpreserves the previously tested file tree exactly.