Add a deterministic NCCL backend for order-stable collectives

[fzyzcjy]

Add an opt-in deterministic NCCL process-group backend (--debug-deterministic-collective)
that folds order-sensitive SUM/AVG reductions into a fixed order so training
collectives are bit-reproducible, registering it as the training world's
distributed backend and requiring synchronous grad sync.

• det_process_group.py (+ GPU test, dist test helper).
• train_actor.py: register the backend and select it when enabled.
• initialize.py: assert synchronous grad reduce under the deterministic backend.

[gemini-code-assist]

Code Review

This pull request introduces a new det_nccl backend to enable bitwise-deterministic SUM and AVG reductions using a fixed-order fold over all-gather operations, along with integration into Megatron initialization, Ray train actors, and comprehensive tests. The review feedback suggests several key improvements: optimizing the pairwise tree reduction by using in-place addition (.add_) to reduce memory overhead, refining the process group type check to prevent native PyTorch groups from bypassing the efficient _allgather_base path, and using getattr when accessing command-line arguments to avoid potential AttributeError exceptions when using mock configuration objects.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The current implementation of the pairwise tree reduction for power-of-two world sizes uses out-of-place addition (+), which allocates new tensors at every step of the reduction. For large tensors (such as gradient buffers in MoE models), this can lead to significant temporary memory overhead and potential GPU OOM.

Since the docstring explicitly states that the function may reuse/mutate the gathered buffers as accumulators, we should perform the addition in-place using .add_ to avoid any extra memory allocations.

Suggested change

	while len(partials) > 1:
	partials = [partials[i] + partials[i + 1] for i in range(0, len(partials), 2)]
	while len(partials) > 1:
	partials = [partials[i].add_(partials[i + 1]) for i in range(0, len(partials), 2)]

[gemini-code-assist]

Using isinstance(group, dist.ProcessGroup) to identify wrapper classes (like torchft) is problematic because all native PyTorch process groups (such as ProcessGroupNCCL) also inherit from dist.ProcessGroup. As a result, native process groups will always bypass the highly efficient _allgather_base path and fallback to the slower list-based allgather path in production, introducing unnecessary overhead.

To correctly identify external wrappers while allowing native PyTorch process groups to use _allgather_base, we can check if the group's module does not start with torch.distributed.

Suggested change

	if isinstance(group, dist.ProcessGroup):
	# ProcessGroup wrappers (torchft) inherit ``_allgather_base`` from the C++ base, but it
	# dispatches to a per-device backend they never register; only the overridden list-form
	# ``allgather`` is safe. ``hasattr`` cannot discriminate here.
	rows = list(output.view(group.size(), -1).unbind(dim=0))
	group.allgather([rows], [input], AllgatherOptions()).wait()
	else:
	group._allgather_base(output, input, AllgatherOptions()).wait()
	if isinstance(group, dist.ProcessGroup) and not type(group).__module__.startswith("torch.distributed"):
	# ProcessGroup wrappers (torchft) inherit ``_allgather_base`` from the C++ base, but it
	# dispatches to a per-device backend they never register; only the overridden list-form
	# ``allgather`` is safe. ``hasattr`` cannot discriminate here.
	rows = list(output.view(group.size(), -1).unbind(dim=0))
	group.allgather([rows], [input], AllgatherOptions()).wait()
	else:
	group._allgather_base(output, input, AllgatherOptions()).wait()

[Shi-Dong]

LGTM with a nit.

[Shi-Dong]

Nit: Move this to the beginning?

[yueming-yuan]

nit: should we put in debug_utils?