Pin pyarrow < 25 in Polars tests

[mroeschke]

Description

Polars tests use a feather API that has been deprecated in Pyarrow 25

A fix is in progress on the Polars side pola-rs/polars#28323

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cfe9dff7-b4cf-4837-af94-9ea019682a43

📥 Commits

Reviewing files that changed from the base of the PR and between dda0706 and f8a1919.

📒 Files selected for processing (1)

• ci/test_cudf_polars_polars_tests.sh

---

📝 Walkthrough

Summary by CodeRabbit

• Chores
  • Updated internal test setup to use a compatible PyArrow version.
  • Refreshed licensing information in the test configuration.

Walkthrough

The Polars CI test script updates its SPDX copyright text and pins the cloned Polars development dependency pyarrow to versions below 25 with an explanatory comment.

Changes

Polars CI updates

Layer / File(s)	Summary
Polars CI requirements update ci/test_cudf_polars_polars_tests.sh	Updates the SPDX copyright declaration and replaces the cloned Polars development requirement pyarrow with pyarrow<25, documenting the related warning avoidance.Thanks.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• rapidsai/cudf#22980: Updates related Polars CI scripts, including dependency and test configuration changes.

Suggested labels: cudf-polars

Suggested reviewers: bdice

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the main change: pinning pyarrow below 25 for Polars tests.
Description check	✅ Passed	The description is directly related to the changeset and explains why pyarrow is pinned.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[jameslamb]

This is totally fine, don't bother restarting CI for my suggestion, but want to mention something I think I've said on PRs like this here before (but maybe not to the same audience)... there's an easier way to constrain the solve, that doesn't require regex, sed, or any knowledge of how the polars repo is laid out.

rapids-init-pip sets up a (by default, empty) constraints file at ${PIP_CONSTRAINT} and that's used below. This change would be equivalent:

echo "pyarrow<25" >> "${PIP_CONSTRAINT}"

[mroeschke]

Ah gotcha, thanks for the tip!

Curious, if a pinning in requirements.txt file conflicts with a pip constraint e.g. pyarrow>=25 with a pyarrow < 25 constraint, is the constraint ignored? The docs weren't really clear on this case https://pip.pypa.io/en/stable/user_guide/#constraints-files

[jameslamb]

The solve will fail in that case

$ echo "pip>25" > ./requirements.txt
$ echo "pip<25" > ./constraints.txt
$ pip install -r ./requirements.txt -c ./constraints.txt
ERROR: Cannot install pip>25 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested pip>25
    The user requested (constraint) pip<25

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

At least, with recent versions of pip. I only tested with what I had laying around

$ pip --version
pip 24.0 from /usr/lib/python3/dist-packages/pip (python 3.12)

Another way to think about this... pip 's solve is "install all the requirements, while not violating any constraints".

That means:

• constraints on packages that never end up actually getting installed are generally harmless
• you can have many requirements and constraints referencing the same package, as long as there's at least one package available that satisfies all of them

So freely >>-ing constraints onto constraints.txt is a great way to constraint the solve, and you don't need to care about the exact state of that file or any other requirements files / requirements from wheels.

[KyleFromNVIDIA]

Shouldn't it be >> rather than <<? Addressed in #23218 (comment)

[jameslamb]

ha yes that was a typo, fixed. Thank you for catching that.

[mroeschke]

/merge