[https://nvbugs/6262973][feat] Pool NCCL window tensor outputs - #1
Open
nv-lschneider wants to merge 9 commits into
Open
[https://nvbugs/6262973][feat] Pool NCCL window tensor outputs#1nv-lschneider wants to merge 9 commits into
nv-lschneider wants to merge 9 commits into
Conversation
Signed-off-by: Ludwig Schneider <lschneider@nvidia.com>
Signed-off-by: Ludwig Schneider <lschneider@nvidia.com>
Signed-off-by: Ludwig Schneider <lschneider@nvidia.com>
Signed-off-by: Ludwig Schneider <lschneider@nvidia.com>
nv-lschneider
force-pushed
the
graph-sage-symmetric
branch
from
August 25, 2026 13:01
d9d5957 to
cc1fe73
Compare
Signed-off-by: Ludwig Schneider <lschneider@nvidia.com>
Signed-off-by: Ludwig Schneider <lschneider@nvidia.com>
Support caller-provided NVFP4 outputs across CUTLASS, cuBLASLt, CUDA-core, CuteDSL, and Marlin backends while preserving unified autotuning. Register the output variants with compilation metadata and fake implementations, enable the validated FP8-QDQ pooling policy, and expand backend correctness coverage.
nv-lschneider
force-pushed
the
perf/nccl-window-output-tensor-pool-only-review
branch
from
August 25, 2026 14:48
74e8b74 to
27e573a
Compare
nv-lschneider
force-pushed
the
graph-sage-symmetric
branch
5 times, most recently
from
September 3, 2026 17:50
12ea73e to
14a045a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@coderabbitai summary
Description
Reuse engine-owned NCCL-window storage for eligible row-parallel GEMM outputs,
allowing the following all-reduce to consume the GEMM result directly.
The pool keeps a stable allocation for warmup and CUDA-graph capture, normalizes
its capacity across TP ranks, and falls back to the allocating path when a later
shape exceeds the retained capacity.
Performance improvements were observed for NVFP4 Llama 3.3 70B and FP8
QDQ/cuBLAS Llama 3.3 70B. FP8 rowwise Llama 3.1 8B was also exercised.
The
cublas_mmpath remains explicit opt-in because it did not show a benefitfor GPT-OSS 20B.
Test Coverage
NCCL_SYMMETRIC.cublas_mm: exercised as an explicit opt-in path.PR Checklist