Skip to content

Ensure librapidsmpf supports the same UCX version range as libucxx #1271

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rapids-bot merged 7 commits into from
Oct 22, 2026
Merged

Ensure librapidsmpf supports the same UCX version range as libucxx #1271

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f3e2234
Ignore ucx run_exports in librapidsmpf
TomAugspurger Oct 21, 2026
61a4085
Add explicit ucx run requirement to librapidsmpf
TomAugspurger Oct 21, 2026
dc145dc
Ignore ucx run_exports in librapidsmpf-tests too
TomAugspurger Oct 21, 2026
b53d7e7
Raise shuffler test wait timeouts to 120s
TomAugspurger Oct 21, 2026
9950333
Revert "Raise shuffler test wait timeouts to 120s"
nirandaperera Oct 21, 2026
68a40e4
Skip RejectsZeroPinnedMaxPoolSize when pinned memory is unsupported
nirandaperera Oct 21, 2026
45f876e
drop 'ucx' from host environment
jameslamb Oct 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions conda/recipes/librapidsmpf/recipe.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,6 @@ outputs:
- libpmix-devel >=5.0,<6.0
- librmm =${{ minor_version }}
- libucxx ${{ ucxx_version }}
- ucx

@jameslamb jameslamb Oct 22, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just pushed 45f876e removing ucx from the host environment.

Here's what I think is happening:

  1. Upgrade conda builds to latest rattler-build #1258 added ucx to the host environment explicitly
  2. as a result, ucx's run_exports make it into librapidsmpf's runtime dependencies (run_exports only get triggered if you explicitly list something in a recipe)
  3. at that time, the latest was v1.22.0, so that got pulled in, resulting in this:
│ ucx              ┆ >=1.22.0,<1.23.0a0 (RE of [host: ucx])                │

(build link)

  1. In Exclude ucx=1.22.0 from supported versions. ucxx#784, libucxx explicitly added a !=1.22.0 to skip that ucx version
  2. sad, now anything depending on librapidsmpf and libucxx has to fall back to older versions of one of them because they have incompatible requirements (<- bad and not sustainable!)

We should be grateful, this is actually a packaging bug. It meant that librapidsmpf and anything depending on it would have had a constraint of "only very-new UCX", so e.g. you couldn't create an environment with cudf-polars and ucx=1.20 or similar.

@jameslamb jameslamb Oct 22, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far, this is looking better 😁

librapidsmpf packages getting a ucx * pin (so no new constraint, totally relying on whatever libucxx supports):

 Finalized run dependencies (librapidsmpf-26.12.00a57-cuda12_261022144244_45f876e1):
 ╭──────────────────┬───────────────────────────────────────────────╮
 │ Name             ┆ Spec                                          │
 ╞══════════════════╪═══════════════════════════════════════════════╡
 │ Run dependencies ┆                                               │
 │ cuda-cupti       ┆ *                                             │
 │ cuda-version     ┆ >=12,<13.0a0 (PC)                             │
...
 │ libucxx          ┆ 0.53.*                                        │
 │                  ┆ >=0.53.0a28,<0.54.0a0 (RE of [host: libucxx]) │
 │                  ┆ >=0.53.0a28,<0.54.0a0 (RE of [host: libucxx]) │
 │ openmpi          ┆ >=5.0                                         │
 │ ucx              ┆ *                                             │
 ╰──────────────────┴───────────────────────────────────────────────╯

(conda-cpp-build link)

- openmpi >=5.0 # See <https://github.com/rapidsai/rapidsmpf/issues/17>

- package:
Expand Down Expand Up @@ -134,6 +133,7 @@ outputs:
- librmm =${{ minor_version }}
- libpmix >=5.0,<6.0
- libucxx ${{ ucxx_version }}
- ucx # librapidsmpf.so links libucp; the version range comes from libucxx
- openmpi >=5.0 # See <https://github.com/rapidsai/rapidsmpf/issues/17>
ignore_run_exports:
from_package:
Expand Down Expand Up @@ -190,7 +190,7 @@ outputs:
- librmm =${{ minor_version }}
- openmpi >=5.0 # See <https://github.com/rapidsai/rapidsmpf/issues/17>
- libucxx ${{ ucxx_version }}
- ucx
- ucx # bare pin to suppress overlinking errors (rely on libucxx's range)
ignore_run_exports:
from_package:
- ${{ compiler("c") }}
Expand Down
4 changes: 4 additions & 0 deletions cpp/tests/test_buffer_resource.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,10 @@ TEST(BufferResource, HostMemoryAvailabilityTracksLiveAllocations) {
}

TEST(BufferResource, RejectsZeroPinnedMaxPoolSize) {
if (!is_pinned_memory_resources_supported()) {
GTEST_SKIP() << "Pinned memory not supported on this system";
}

rmm::mr::cuda_memory_resource cuda_mr;
EXPECT_THROW(
std::ignore = BufferResource::create(
Expand Down
Loading