[rakhmets]

What?

Disabled UCX CUDA IPC cache.

Why?

The previous fix of memory leak (#1846) was reverted (#1996).

These changes (openucx/ucx#11288, openucx/ucx#11730) help eliminate the memory leak by disabling CUDA IPC cache.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 835d84e3-3bea-4100-b678-68e0c6ada075

📥 Commits

Reviewing files that changed from the base of the PR and between 2301000 and 5c47ab4.

📒 Files selected for processing (1)

• examples/device/ep/csrc/nixl_ep.cpp

---

📝 Walkthrough

Walkthrough

The NIXL UCX backend initialization now sets engine_config to disable the CUDA IPC cache.

Changes

UCX configuration

Layer / File(s)	Summary
Disable CUDA IPC cache examples/device/ep/csrc/nixl_ep.cpp	The UCX backend initialization sets engine_config = "CUDA_IPC_CACHE=n".

Estimated code review effort: 1 (Trivial) | ~2 minutes

Merge Risk: ⚪ Minimal · up to 5c47a

This is a narrowly scoped change to disable the CUDA IPC cache, with no actionable merge-blocking risk remaining beyond normal checks and review.

Suggested reviewers: ebarilanm, itayalroy

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely identifies the main change: disabling the UCX CUDA IPC cache.
Description check	✅ Passed	The description includes the required What and Why sections and explains the memory-leak motivation; the optional How section is not needed for this simple change.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[rakhmets]

/build

[itayalroy]

IIUC, connect_ranks relies on the UCX CUDA IPC cache, it destroys the old mview and creates a new one assuming the NVLink ptrs exported during the creation of the old mview are still valid. We cannot disable the cache until we get rid of this assumption. I'm already working on it so should have a PR up soon, so I think we should wait with merging #2089 until then

[rakhmets]

IIUC, connect_ranks relies on the UCX CUDA IPC cache, it destroys the old mview and creates a new one assuming the NVLink ptrs exported during the creation of the old mview are still valid. We cannot disable the cache until we get rid of this assumption. I'm already working on it so should have a PR up soon, so I think we should wait with merging #2089 until then

Do you mean that there will be an issue if connect_ranks is called, and NIXL EP still runs a CUDA kernel, which uses inter process memory handles opened on previous iteration/stage?

[itayalroy]

@LinoyGeva and I are seeing intermittent NIXL EP timeouts with vLLM Elastic EP when UCX_CUDA_IPC_CACHE=n which do not reproduce with UCX_CUDA_IPC_CACHE=y.

vLLM also uses the NIXL host API for EPLB weight transfers, so this environment variable disables the IPC cache for the separate EPLB agent as well. We suspect the timeouts are related to that. We cannot merge this PR until we fix this issue.

I think it will be easier to fix the memory leak with UCX_CUDA_IPC_CACHE=y first as we discussed @yosefe @ofirfarjun7 @rakhmets