[II] Pack DCP attention LSE as FP32 - #339
voipmonitor wants to merge 1 commit into
Conversation
Normalize activation-dtype LSE tensors to FP32 before the DCP all-to-all pack kernel and preserve their FP32 bit pattern in the output-dtype transport lanes. The unpacked global LSE remains FP32. The communication topology, output activation dtype, and weighted-combination formula are unchanged. Validation: Ruff format/check and git diff --check pass. Thirteen pack/unpack tests pass across FP16, BF16, and FP32 output buffers. Four 4-GPU NCCL tests pass with and without the vLLM workspace manager.
|
Warning Review limit reached
Next review available in: 26 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 participant
Status
Qualified.
Behavior
The DCP all-to-all path converts FP16 or BF16 log-sum-exp tensors to FP32 before packing. The transport preserves each FP32 LSE bit pattern in one FP32 lane or two 16-bit output-dtype lanes, and unpacking returns the global LSE in FP32.
Technical reason
MLA backends may emit LSE in the activation dtype, while the packed DCP protocol interprets the payload as FP32 bits. Normalizing before the Triton pack kernel gives every backend the same wire representation and prevents dtype-dependent bit reinterpretation.
Compatibility
The communication topology, output activation dtype, LSE-weighted combination formula, B12X dispatch, and NCCL fallback are unchanged. FP32 LSE inputs avoid an allocation and retain their existing representation.
Validation
git diff --checkpasses.