[cudf_streaming] Using rapidsmpf spill unspill partition

[nirandaperera]

Description

Removed spill/unspill_partitions from partition.hpp and use rapidsmpf utils.

Depends on rapidsai/rapidsmpf#1087

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: fcb23ba3-6f72-4d3b-81d8-2f4f28c55696

📥 Commits

Reviewing files that changed from the base of the PR and between c66c343 and fe10bb8.

📒 Files selected for processing (3)

• python/cudf_streaming/cudf_streaming/integrations/partition.pxd
• python/cudf_streaming/cudf_streaming/integrations/partition.pyi
• python/cudf_streaming/cudf_streaming/integrations/partition.pyx

🚧 Files skipped from review as they are similar to previous changes (2)

• python/cudf_streaming/cudf_streaming/integrations/partition.pyi
• python/cudf_streaming/cudf_streaming/integrations/partition.pyx

---

📝 Walkthrough

Summary by CodeRabbit

Release Notes

• Refactor

  • Removed spill_partitions and unspill_partitions public functions from the partition integration API (C++ and Python). Users relying on these functions will need to update their code.

• Tests

  • Removed spill/unspill roundtrip tests and associated test utilities.

Walkthrough

Removes local implementations and Python bindings for spill_partitions/unspill_partitions from libcudf_streaming, updates C++ call sites to call rapidsmpf::unspill_partitions, removes related tests/helpers, and tightens Cython imports to match the reduced API.

Changes

Spill/Unspill API Consolidation

Layer / File(s)	Summary
C++ public API removal cpp/libcudf_streaming/include/cudf_streaming/integrations/partition.hpp, cpp/libcudf_streaming/src/integrations/partition.cpp	Declarations and implementations of spill_partitions and unspill_partitions are removed; unpack_and_concat remains as the integrations API.
C++ call-site migration to RAPIDSMPF cpp/libcudf_streaming/src/streaming/partition.cpp	Adds #include <rapidsmpf/memory/spill.hpp> and changes the call to use rapidsmpf::unspill_partitions inside unpack_and_concat.
Test helper and fixture removal cpp/libcudf_streaming/tests/utils.hpp, cpp/libcudf_streaming/tests/test_partition.cpp	Removes create_packed_data helper and the spilling test fixture/roundtrip test that exercised spill/unspill.
Python/Cython binding removal python/cudf_streaming/cudf_streaming/integrations/partition.pxd, .../partition.pyi, .../partition.pyx	Removes Cython cpdef declarations and extern bindings for spill_partitions/unspill_partitions, and trims RAPIDSMPF cimports (AllowOverbooking, packed_data_vector_to_list).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

• Follow-ups for rapidsmpf to cudf migration #22771: Addresses the proposal to move spill/unspill functions back to RAPIDSMPF by removing duplicate local implementations.

Possibly related PRs

• rapidsai/cudf#22760: Previously added the Python/Cython bindings for spill_partitions/unspill_partitions; this PR removes them as part of consolidation.

Suggested labels

libcudf, Python

Suggested reviewers

• galipremsagar
• mroeschke
• bdice
• jameslamb

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change: removing spill/unspill_partitions from cudf_streaming and replacing them with rapidsmpf utilities.
Description check	✅ Passed	The description is directly related to the changeset, explaining the removal of spill/unspill_partitions and replacement with rapidsmpf utils, and notes a dependency on rapidsmpf PR.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[pentschev]

/merge