Minimize Ray resource acquisition in cudf_polars tests

[mroeschke]

Description

Similar to #22646 in approach to minimize Ray resource spin-up in cudf_polars tests

• Removes test_scan, test_filter, test_group_by, test_join, and test_empty_dataframe as they already have coverage in other test where we parameterize over engine
• For tests needing an instantiated DaskEngine, uses the session-scoped DaskEngine already created in conftest.py
• Downsizing the Ray cluster's default parameters by sharing a ray_init_options fixture for initialization
  • num_cpus set to 2 instead of auto-detecting virtual cores
  • num_gpus set to 0 (IIUC this detection is only needed by the workers)
  • include_dashboard always False
  • object_store_memory set to 256 MB instead of min(0.3 * system memory, 200GB)

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e6a4415d-ba48-488c-b718-12602e58a92f

📥 Commits

Reviewing files that changed from the base of the PR and between ec6af5b and 474a3cd.

📒 Files selected for processing (1)

• python/cudf_polars/tests/streaming/test_ray.py

🚧 Files skipped from review as they are similar to previous changes (1)

• python/cudf_polars/tests/streaming/test_ray.py

---

📝 Walkthrough

Summary by CodeRabbit

• Tests
  • Made streaming engine test configuration session-scoped and configurable (cluster size and init options).
  • Refactored tests to use a shared, configurable Ray test setup instead of per-module/local engine instances; updated GPU, shutdown, and reset tests to use the shared fixtures and parameters.

Walkthrough

Refactors Ray test setup to use session-scoped ray_num_ranks and ray_init_options fixtures in conftest.py, wires those into RayEngine construction and pytest_generate_tests, and updates test_ray.py to consume the shared ray_engine and Ray configuration fixtures.

Changes

Ray Engine Configuration Parameterization

Layer / File(s)	Summary
Ray configuration fixtures and engine wiring python/cudf_polars/tests/conftest.py	Adds Any import, introduces ray_num_ranks() and ray_init_options(ray_num_ranks) session-scoped fixtures, extends _unconfigured_engine to accept these fixtures, constructs RayEngine with num_ranks=ray_num_ranks and forwards ray_init_options, adds a ray_engine fixture, and updates pytest_generate_tests to parametrize only ["ray"] when ray_engine is requested`.
Test module migration to shared fixtures python/cudf_polars/tests/streaming/test_ray.py	Removes module-level NUM_RANKS and local engine fixture, updates tests to accept ray_engine, ray_num_ranks, and ray_init_options, parameterizes reset_engine to accept Ray configuration, and updates shutdown/reset tests to construct engines using the new fixtures.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• rapidsai/cudf#22493: Refactors tests/conftest.py engine configuration pipeline that forms the foundation for this PR's Ray-specific fixture additions.

Suggested reviewers

• rjzamora
• jameslamb

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title directly and clearly describes the main objective of the pull request—minimizing Ray resource acquisition in cudf_polars tests.
Description check	✅ Passed	The description is well-related to the changeset, providing context about the Ray resource optimization approach, specific configuration changes, and test removals.
Docstring Coverage	✅ Passed	Docstring coverage is 95.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[madsbk]

Thanks @mroeschke

[mroeschke]

/merge