Skip to content
This repository was archived by the owner on Oct 6, 2026. It is now read-only.

Retain Trellis route-pack programs in prepared MoE plans - #442

Merged
lukealonso merged 1 commit into
masterfrom
fix/trellis-prepared-route-pack
Oct 1, 2026
Merged

lukealonso merged 1 commit into
masterfrom
fix/trellis-prepared-route-pack

Conversation

@voipmonitor

Copy link
Copy Markdown
Contributor

Behavior

Retains the compiled Trellis route-pack launch carriers when materializing a prepared fused-MoE plan. The materialized caller-owned scratch receives the same carriers produced during preparation.

Without those carriers, a different live row count can try to resolve a packing kernel after kernel resolution has been frozen for serving. This is an ownership fix; it does not change expert selection, route order, decoded weights or output arithmetic.

Implemented and qualified. Targets master directly. It is independent of the optional stable-packing algorithm in #437: both the ordinary packing path and the full-codeword decoder can use this fix.

Validation

python -m pytest tests/moe/test_exl3_prepare.py -k canonical_exl3_preparation -q

24 GPU cases passed: capacities 1/8/129, uneven source extents, SiLU/SiTU, changed live row counts under frozen resolution, carrier identity, numerical reference, poisoned scratch, and CUDA Graph replay with stable allocations.

Kimi-K3 EXL3 TP9/DCP9 with Red Hat DSpark K5 starts and reproduces the control's three 4096-token outputs and acceptance counters. The integration details are in vLLM #906.

The existing preparation-ownership PR #384 concerns callable owner discovery and collective initialization; it does not transfer this component's route-pack field. AI assistance: OpenAI Codex. Maintainer review is required.

Keep the compiled route pack in the materialized scratch plan so bindings do not enter Triton JIT resolution. Preserve numerical behavior. Validate 24 GPU extent, activation and capacity cases including frozen resolution and poisoned-scratch graph replay.
@coderabbitai

coderabbitai Bot commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 44 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: local-inference-lab/b12x/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 6f995a69-2b8b-4f9f-9098-1ae4b69460e2

📥 Commits

Reviewing files that changed from the base of the PR and between b4b12bc and efb9149.

📒 Files selected for processing (3)
  • .lil/changes/b12x-trellis-route-pack-ownership.json
  • b12x/moe/fused_moe/_preparation.py
  • tests/moe/test_exl3_prepare.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants