Adding cudf streaming dist tests

[nirandaperera]

Description

Adds multi-rank C++ test executables for cudf_streaming that exercise the test suite under MPI and UCXX communicators, mirroring rapidsmpf's mpi_tests/ucxx_tests.

Changes

• cmake/ConfigureOptionalCommunication.cmake: Enable MPI/UCXX detection under BUILD_TESTS (previously only BUILD_BENCHMARKS/BUILD_EXAMPLES), so CUDF_STREAMING_HAVE_MPI/CUDF_STREAMING_HAVE_UCXX are set for test builds.
• cmake/thirdparty/get_cudf.cmake: Request cudf's optional testing component when BUILD_TESTS is on, so cudf::cudftestutil{,_impl} resolve from the installed (conda) cudf package without a local cudf build.
• tests/environment.hpp: Refactor Environment into an abstract base exposing a polymorphic communicator interface; add MPI/UCXX to TestEnvironmentType. Communicator-specific state (e.g. mpi_comm_) moves into the concrete subclasses.
• tests/main/mpi.cpp (new): MPIEnvironment that initializes MPI, builds a rapidsmpf::MPI communicator, and provides barrier()/split_comm().
• tests/main/ucxx.cpp (new): UCXXEnvironment that bootstraps UCXX over MPI (init_using_mpi) with the required CUDA-context/thread-level setup.
• tests/main/single.cpp: Convert to a SingleEnvironment subclass of the new abstract base.
• tests/CMakeLists.txt: Add a cudf_streaming_mpirun_test_add helper and build cudf_streaming_mpi_tests/cudf_streaming_ucxx_tests (gated on CUDF_STREAMING_HAVE_MPI/UCXX), registering per-rank mpirun ctest cases; apply CUDF_CUDA_FLAGS to the shared test sources so the injected cudf testutil .cu files compile.

depends on #22860

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

This PR extends CMake discovery for BUILD_TESTS, refactors Environment into an abstract base class, adds mpirun test registration and CUDA compile handling, and implements Single, MPI, and UCXX test environments with corresponding test entrypoints.

Changes

Test environment setup

Layer / File(s)	Summary
CMake dependency discovery for tests cpp/libcudf_streaming/cmake/ConfigureOptionalCommunication.cmake, cpp/libcudf_streaming/cmake/thirdparty/get_cudf.cmake	MPI/UCXX discovery conditions now include BUILD_TESTS; cudf's testing component is conditionally found when tests are enabled and cudf::cudftestutil is missing.
Abstract Environment base class cpp/libcudf_streaming/tests/environment.hpp	Converts Environment into an abstract base with inline constructor/destructor and pure-virtual SetUp, TearDown, barrier, type, and split_comm; extends TestEnvironmentType to include MPI and UCXX.
CMake test infrastructure and targets cpp/libcudf_streaming/tests/CMakeLists.txt	Adds cudf_streaming_mpirun_test_add helper to register multi-rank (-np 1, 2, 5) ctest entries; forwards ${CUDF_CUDA_FLAGS} only for CUDA sources; conditionally creates cudf_streaming_mpi_tests and cudf_streaming_ucxx_tests linking different environment mains.
SingleEnvironment refactor cpp/libcudf_streaming/tests/main/single.cpp	Implements SingleEnvironment subclass overriding the abstract interface, manages rapidsmpf::Single lifecycle, and instantiates it as the global test environment in main().
MPIEnvironment implementation cpp/libcudf_streaming/tests/main/mpi.cpp	Adds MPIEnvironment with MPI init/finalize, duplicated MPI_COMM_WORLD handling, MPI barrier, and lazy MPI_Comm_split creation returning shared_ptr with custom deleter; registers MPI main().
UCXXEnvironment implementation cpp/libcudf_streaming/tests/main/ucxx.cpp	Adds UCXXEnvironment that enforces CUDA context creation ordering, uses MPI_Init_thread, creates UCXX communicator, provides barrier and cached split communicator, and registers UCXX main().

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

• rapidsai/cudf#22771: This PR directly implements the MPI and UCXX test environments and CMake configuration for libcudf_streaming testing, resolving the follow-up work item that references this change.

Possibly related PRs

• rapidsai/cudf#22860: Both PRs modify cpp/libcudf_streaming/cmake/ConfigureOptionalCommunication.cmake to conditionally discover MPI/UCXX support, with this PR extending the BUILD_* conditions to include BUILD_TESTS.
• rapidsai/cudf#22853: Both PRs extend CMake logic to enable MPI/UCXX-dependent targets under BUILD_BENCHMARKS, with this PR additionally gating test targets on optional dependency availability.

Suggested reviewers

• bdice
• pentschev
• KyleFromNVIDIA
• madsbk

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Adding cudf streaming dist tests' directly describes the main change—introducing distributed (multi-rank) test executables for cudf_streaming under MPI and UCXX communicators, which is the central objective of this PR.
Description check	✅ Passed	The description comprehensively details the changes across all modified and new files, explaining the rationale (mirroring rapidsmpf's mpi_tests/ucxx_tests pattern) and the technical approach (refactoring Environment into an abstract base, adding MPI/UCXX support, and configuring CMake to enable optional dependencies for tests).
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cpp/libcudf_streaming/cmake/thirdparty/get_cudf.cmake`:
- Around line 19-20: The current guard only skips find_package when
cudf::cudftestutil exists, but tests also need cudf::cudftestutil_impl; update
the logic in get_cudf.cmake so the testing-component guard covers both imported
targets (check NOT TARGET cudf::cudftestutil AND NOT TARGET
cudf::cudftestutil_impl before skipping) or alternatively always call
find_package(cudf ... COMPONENTS testing) when BUILD_TESTS is true and then
validate that both cudf::cudftestutil and cudf::cudftestutil_impl are available
after the call, erroring (or re-invoking find_package) if either is missing.

In `@cpp/libcudf_streaming/tests/main/mpi.cpp`:
- Around line 48-68: split_comm() currently creates and returns a fresh
shared_ptr but never stores it into the member split_comm_, causing repeated
MPI_Comm_split calls and losing ownership for teardown; fix by assigning the
newly created shared_ptr to split_comm_ (i.e. split_comm_ =
std::shared_ptr<rapidsmpf::MPI>(...)) before returning it, and ensure the custom
deleter captures the split MPI_Comm handle you created so MPI_Comm_free is
invoked exactly once when split_comm_ is destroyed (use the local split_comm
variable captured by value in the deleter). Ensure the early-return check in
split_comm() still returns the cached split_comm_ when present.

In `@cpp/libcudf_streaming/tests/main/ucxx.cpp`:
- Around line 31-32: The call to cudaFree(nullptr) used to force CUDA context
creation must check and handle the CUDA API return status; update the
initialization code that currently calls cudaFree(nullptr) so it captures the
returned cudaError_t, tests for cudaSuccess, and on failure logs the CUDA error
string (cudaGetErrorString) and aborts/returns non-zero before proceeding to
UCX/MPI initialization (i.e., change the unchecked cudaFree(nullptr) call in
ucxx.cpp to check the result and exit with an error message if initialization
fails).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 47aa4303-d984-42fe-bf6d-2df22cebb835

📥 Commits

Reviewing files that changed from the base of the PR and between 0b8ef6a and 7a6bb81.

📒 Files selected for processing (7)

• cpp/libcudf_streaming/cmake/ConfigureOptionalCommunication.cmake
• cpp/libcudf_streaming/cmake/thirdparty/get_cudf.cmake
• cpp/libcudf_streaming/tests/CMakeLists.txt
• cpp/libcudf_streaming/tests/environment.hpp
• cpp/libcudf_streaming/tests/main/mpi.cpp
• cpp/libcudf_streaming/tests/main/single.cpp
• cpp/libcudf_streaming/tests/main/ucxx.cpp

[vyasr]

Looks OK

[coderabbitai]

🧹 Nitpick comments (1)

cpp/libcudf_streaming/tests/environment.hpp (1)

38-38: 📐 Maintainability & Code Quality | 💤 Low value

Consider adding [[nodiscard]] to options().

The accessor is side-effect-free and returns a reference; discarding it is almost certainly a bug.

Suggested change

-  rapidsmpf::config::Options& options() { return options_; }
+  [[nodiscard]] rapidsmpf::config::Options& options() { return options_; }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cpp/libcudf_streaming/tests/environment.hpp` at line 38, The accessor
options() returns a reference and is side-effect-free, so add the [[nodiscard]]
attribute to its declaration to prevent callers from accidentally discarding the
returned reference; update the signature from "rapidsmpf::config::Options&
options() { return options_; }" to include the attribute (e.g. "[[nodiscard]]
rapidsmpf::config::Options& options() { return options_; }") so the compiler
will warn when the result is ignored.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@cpp/libcudf_streaming/tests/environment.hpp`:
- Line 38: The accessor options() returns a reference and is side-effect-free,
so add the [[nodiscard]] attribute to its declaration to prevent callers from
accidentally discarding the returned reference; update the signature from
"rapidsmpf::config::Options& options() { return options_; }" to include the
attribute (e.g. "[[nodiscard]] rapidsmpf::config::Options& options() { return
options_; }") so the compiler will warn when the result is ignored.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 606505fb-30fb-463d-8809-becece25af85

📥 Commits

Reviewing files that changed from the base of the PR and between e0cbaec and cef0222.

📒 Files selected for processing (1)

• cpp/libcudf_streaming/tests/environment.hpp

[pentschev]

One nit about binary naming, otherwise LGTM.

[pentschev]

Suggested change

	add_executable(cudf_streaming_mpi_tests main/mpi.cpp)
	add_executable(libcudf_streaming_mpi_tests main/mpi.cpp)

[pentschev]

Nit: let's make test names complete and matching the package/library name. This would also match the same prefix being proposed for benchmarks here.

[nirandaperera]

/ok to test b851330

[nirandaperera]

/ok to test 41bb34e