[II] Gather paired DCP projections behind one barrier - #342
voipmonitor wants to merge 1 commit into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
7ac5a6b to
db4765b
Compare
Expose a vLLM binding for B12X DcpAllToAllPool.all_gather_pair. Two contiguous rank-local projection rows share one IPC synchronization epoch when their row widths satisfy the 16-byte transport contract. Unsupported dtypes, shapes, capacities, alignments, or transports use two exact process-group gathers in input order. Validation: Ruff format/check and git diff --check pass. Focused tests cover DCP16 paired dispatch, mixed BF16/FP32 rows, alignment rejection, exact fallback ordering, and DCP16 warmup.
adb9582 to
2846390
Compare
|
The behavior implemented by this pull request is preserved in vLLM #384 as commit |
Status
Qualified.
Behavior
dcp_b12x_all_gather_pairgathers two contiguous rank-local projection rows through one B12X IPC synchronization epoch. FP16, BF16, FP32, and FP8 E4M3 rows are accepted when each row occupies a multiple of 16 bytes.Unsupported shapes, dtypes, capacities, alignments, world sizes, or B12X availability use two exact process-group gathers in the original tensor order.
Technical reason
Decode pipelines often gather a latent projection and a router projection together. Separate collectives pay two system-scope barrier epochs even though both tensors have the same batch and process group. B12X already exposes a paired gather that shares the barrier.
Compatibility
The public helper has an exact fallback and does not change existing single-tensor gather behavior. The change depends on the DCP16 dispatch guard in #338. B12X
DcpAllToAllPool.all_gather_pairalready provides the transport primitive.Validation
git diff --checkpasses.