Revert "Remove unnecessary barrier in share mem handle cuda ipc"

[Priya2698]

Reverts #5260

RingAllgatherOverlapTest.RingAllgatherBasedPipeliningHostIRImplementationCudaIpc failed for this PR.

[Priya2698]

!test

[github-actions]

Description

• Reintroduce barrier to ensure memhandle synchronization

• Prevent correctness issues in IPC handle exchange

• Fix failing RingAllgatherOverlapTest test case

• Ensure ranks synchronize before next handle exchange

---

Changes walkthrough 📝

	Relevant files
Bug fix	ipc_handle.cpp Reintroduce synchronization barrier in IPC handle exchange csrc/multidevice/ipc_handle.cpp Readded barrier after inserting IPC handles Ensures all ranks receive handles before proceeding Prevents premature key deletion from store Fixes correctness issue in handle exchange +0/-3

---

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 No relevant tests

⚡ Recommended focus areas for review Missing Early Return The reverted code removes the early return condition when non_cached_communications is empty, which may lead to unnecessary barrier execution and potential performance impact. This change should be validated for correctness and performance implications. auto ipc_handles = std::make_unique<P2pIpcHandle>( std::move(local_ipc_handle), std::move(peer_ipc_handle)); insert(communication, std::move(ipc_handles)); } // a barrier is needed here to ensure all ranks have received the // memhandles and the keys are deleted from the store before the next call to // exchangeHandles, otherwise there is a correctness issue // TODO: precisely select what ranks need to wait on that barrier.

[samnordmann]

Thanks, and sorry for the trouble. I believe the allocation cache #4466 should fix the issue. I'll reopen a new PR later