Skip to content

Pin pyarrow < 25 in Polars tests #23218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rapids-bot merged 1 commit into from
Jul 10, 2026
Merged

Pin pyarrow < 25 in Polars tests #23218

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion ci/test_cudf_polars_polars_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -euo pipefail
Expand Down Expand Up @@ -43,6 +43,8 @@ git clone https://github.com/pola-rs/polars.git --branch "${TAG}" --depth 1
rapids-logger "Install polars test requirements"
# We don't need to pick up dependencies from polars-cloud, so we remove it.
sed -i '/^polars-cloud$/d' polars/py-polars/requirements-dev.txt
# Pin pyarrow to avoid FutureWarning https://github.com/pola-rs/polars/pull/28323.
sed -i 's/^pyarrow$/pyarrow<25/' polars/py-polars/requirements-dev.txt

@jameslamb jameslamb Jul 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is totally fine, don't bother restarting CI for my suggestion, but want to mention something I think I've said on PRs like this here before (but maybe not to the same audience)... there's an easier way to constrain the solve, that doesn't require regex, sed, or any knowledge of how the polars repo is laid out.

rapids-init-pip sets up a (by default, empty) constraints file at ${PIP_CONSTRAINT} and that's used below. This change would be equivalent:

echo "pyarrow<25" >> "${PIP_CONSTRAINT}"

@mroeschke mroeschke Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah gotcha, thanks for the tip!

Curious, if a pinning in requirements.txt file conflicts with a pip constraint e.g. pyarrow>=25 with a pyarrow < 25 constraint, is the constraint ignored? The docs weren't really clear on this case https://pip.pypa.io/en/stable/user_guide/#constraints-files

@jameslamb jameslamb Jul 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The solve will fail in that case

$ echo "pip>25" > ./requirements.txt
$ echo "pip<25" > ./constraints.txt
$ pip install -r ./requirements.txt -c ./constraints.txt
ERROR: Cannot install pip>25 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested pip>25
    The user requested (constraint) pip<25

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

At least, with recent versions of pip. I only tested with what I had laying around

$ pip --version
pip 24.0 from /usr/lib/python3/dist-packages/pip (python 3.12)

Another way to think about this... pip 's solve is "install all the requirements, while not violating any constraints".

That means:

  • constraints on packages that never end up actually getting installed are generally harmless
  • you can have many requirements and constraints referencing the same package, as long as there's at least one package available that satisfies all of them

So freely >>-ing constraints onto constraints.txt is a great way to constraint the solve, and you don't need to care about the exact state of that file or any other requirements files / requirements from wheels.

@KyleFromNVIDIA KyleFromNVIDIA Jul 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be >> rather than <<? Addressed in #23218 (comment)

@jameslamb jameslamb Jul 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ha yes that was a typo, fixed. Thank you for catching that.


# notes:
#
Expand Down
Loading