Pin Polars<1.43 by Matt711 · Pull Request #22993 · NVIDIA/cudf

Matt711 · 2026-06-25T18:26:38Z

Description

Updated cudf-polars to support Polars 1.42

I have 3-4 follow-up PRs in mind

Fused expression support (should be trivial). See [BUG][Polars 1.41] Polars NodeTraverser does not support IRFunctionExpr::Fused #22797
hive partitioned scans (not trivial in streaming case I think) (may be 2 PRs for in-memory and streaming). See [Story] Support hive-partitioning in cudf-polars #17832
utilize dynamic predicate hint in the sort actor (not trivial)

Checklist

I am familiar with the Contributing Guidelines.
New or existing tests cover these changes.
The documentation is up to date with these changes.

copy-pr-bot · 2026-06-25T18:26:42Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Matt711 · 2026-06-25T18:26:54Z

/ok to test c55949f

Matt711 · 2026-06-29T18:48:39Z

/ok to test f95b02b

Matt711 · 2026-06-29T22:21:06Z

/ok to test 5f9aaf7

Matt711 · 2026-06-29T22:52:52Z

/ok to test 93434d6

Matt711 · 2026-06-30T00:46:28Z

/ok to test 1ad3e5f

Matt711 · 2026-06-30T19:08:50Z

/ok to test 585c4b0

coderabbitai · 2026-06-30T20:40:57Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 40f01018-5df3-4db7-9544-a77ab42a2d5d

📥 Commits

Reviewing files that changed from the base of the PR and between 7c60952 and 909017a.

📒 Files selected for processing (1)

python/cudf_polars/cudf_polars/testing/inject_gpu_engine.py

🚧 Files skipped from review as they are similar to previous changes (1)

python/cudf_polars/cudf_polars/testing/inject_gpu_engine.py

📝 Walkthrough

Summary by CodeRabbit

New Features
- Added a new boolean DSL operation to check whether a column is sorted (including descending and nulls_last behavior).
Bug Fixes
- Improved translation compatibility for newer Polars IR and refined sort/scan handling.
- Added safeguards for hive-partitioned Parquet scans and updated predicate hint behavior.
Tests
- Expanded compatibility coverage up to Polars 1.42 and added new GPU/CPU parity tests (plus updated skips/xfails for known GPU differences).
Chores
- Relaxed Polars version upper bounds (<1.43) across environments and CI/tooling.

Walkthrough

This PR widens Polars version bounds, adds a Polars 1.42 gate constant, updates translation and BooleanFunction handling for newer Polars behavior, and adjusts tests plus expected-failure mappings.

Changes

Polars version and compatibility updates

Layer / File(s)	Summary
Version constraints and compatibility matrix `.pre-commit-config.yaml`, `conda/environments/all_cuda-*.yaml`, `conda/recipes/cudf-polars/recipe.yaml`, `python/cudf_polars/pyproject.toml`, `dependencies.yaml`	Raises the Polars upper bound to `<1.43` across packaging and environment files, and extends the compatibility matrix to include Polars 1.42.
Polars 1.42 version gate `python/cudf_polars/cudf_polars/utils/versions.py`	Adds `POLARS_VERSION_LT_142` and updates the module SPDX header.
IR translation updates `python/cudf_polars/cudf_polars/dsl/translate.py`, `python/cudf_polars/cudf_polars/containers/dataframe.py`	Updates translation imports and version checks, changes dynamic predicate handling, rejects hive-partitioned scans on newer Polars, rewrites sort slice handling, and widens one type annotation.
IsSorted boolean function and tests `python/cudf_polars/cudf_polars/dsl/expressions/boolean.py`, `python/cudf_polars/tests/dsl/test_serialization.py`, `python/cudf_polars/tests/expressions/test_booleanfunction.py`	Adds `IsSorted` support, evaluates it via sortedness checks, and updates serialization and GPU tests for the new enum member and version gate.
Scan tests and expected failures `python/cudf_polars/tests/test_scan.py`, `python/cudf_polars/cudf_polars/testing/inject_gpu_engine.py`, `ci/test_narwhals.sh`	Adds a hive-partitioned scan rejection test, updates a version-gated xfail, and changes injected GPU engine expected failures and narwhals exclusions.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

rapidsai/cudf#22048: Shares the Polars-version-gated translation updates in python/cudf_polars/cudf_polars/dsl/translate.py.
rapidsai/cudf#22189: Also updates the dependencies.yaml compatibility matrix for cuDF-polars and Polars version coverage.
rapidsai/cudf#22605: Related Polars test expectation updates in python/cudf_polars/cudf_polars/testing/inject_gpu_engine.py.
rapidsai/cudf#23020: Similar change to .pre-commit-config.yaml dependency bounds.

Suggested labels: improvement

Suggested reviewers: mroeschke, wence-, bdice

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: widening the Polars upper version bound to below 1.43.
Description check	✅ Passed	The description is clearly related to the changeset and states that cudf-polars was updated to support Polars 1.42.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

🧹 Nitpick comments (1)

python/cudf_polars/tests/expressions/test_booleanfunction.py (1)
298-326: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add edge-case coverage for is_sorted.

Test only covers a 5-element column with at most one null. Consider adding empty, single-element, and all-null cases to exercise plc.sorting.is_sorted boundary behavior.

Based on path instructions, "Ensure test files provide comprehensive edge case coverage (empty, all-null, single-element, mixed types)".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/cudf_polars/tests/expressions/test_booleanfunction.py` around lines
298 - 326, Extend test_boolean_is_sorted to cover the missing boundary cases for
pl.col(...).is_sorted, not just the current 5-element inputs. Add parametrized
assertions for empty, single-element, and all-null Series in the same test
module so the GPUEngine path exercises plc.sorting.is_sorted across these edge
cases, alongside the existing asc/desc/unsorted coverage.
Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/cudf_polars/cudf_polars/testing/inject_gpu_engine.py`:
- Around line 249-251: Scope the xfail entries in inject_gpu_engine.py to the
Polars version that actually triggers them by guarding the affected test names
in the mapping with a version check or version-specific condition. Update the
entries for test_filter_contradiction_fallible_error_handling,
test_hconcat_predicate, and test_union_lazyframe_horizontal so they only apply
when running against Polars 1.42.1, rather than unconditionally disabling those
tests for all versions.

---

Nitpick comments:
In `@python/cudf_polars/tests/expressions/test_booleanfunction.py`:
- Around line 298-326: Extend test_boolean_is_sorted to cover the missing
boundary cases for pl.col(...).is_sorted, not just the current 5-element inputs.
Add parametrized assertions for empty, single-element, and all-null Series in
the same test module so the GPUEngine path exercises plc.sorting.is_sorted
across these edge cases, alongside the existing asc/desc/unsorted coverage.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 765bd2d6-bd98-4070-a616-81d578d85c38

📥 Commits

Reviewing files that changed from the base of the PR and between aa8cfca and cb31390.

📒 Files selected for processing (16)

.pre-commit-config.yaml
conda/environments/all_cuda-129_arch-aarch64.yaml
conda/environments/all_cuda-129_arch-x86_64.yaml
conda/environments/all_cuda-133_arch-aarch64.yaml
conda/environments/all_cuda-133_arch-x86_64.yaml
conda/recipes/cudf-polars/recipe.yaml
dependencies.yaml
python/cudf_polars/cudf_polars/containers/dataframe.py
python/cudf_polars/cudf_polars/dsl/expressions/boolean.py
python/cudf_polars/cudf_polars/dsl/translate.py
python/cudf_polars/cudf_polars/testing/inject_gpu_engine.py
python/cudf_polars/cudf_polars/utils/versions.py
python/cudf_polars/pyproject.toml
python/cudf_polars/tests/dsl/test_serialization.py
python/cudf_polars/tests/expressions/test_booleanfunction.py
python/cudf_polars/tests/test_scan.py

mroeschke

As discussed earlier this week, does this new version allow us to support/close #22797?

Matt711 · 2026-06-30T23:46:50Z

As discussed earlier this week, does this new version allow us to support/close #22797?

Yes it allows us to support it. I have 3-4 follow-up PRs in mind

Fused expression support (should be trivial)
hive partitioned scans (not trivial in streaming case I think) (may be 2 PRs for in-memory and streaming)
utilize dynamic predicate hint in the sort actor (not trivial)

…umn.is_sorted

Matt711 · 2026-07-01T16:59:17Z

/merge

Matt711 · 2026-07-02T17:49:32Z

 # test_dtypes: narwhals' dtype mapping changed with polars 1.40 (reports Object where the test expects Int8).
 # test_namespace_len[polars[lazy]]: len() row count lost in zero-column streaming chunks
 #   (https://github.com/rapidsai/cudf/issues/21428).
+# test_explode_*[polars[lazy]-*]: polars 1.42 emits a DeprecationWarning when explode() is called


I haven't found time TODO #19594 yet. Maybe I'll put an agent on it

Matt711 · 2026-07-02T20:12:23Z

/merge

Pin Polars<1.43

c55949f

Matt711 added feature request New feature or request non-breaking Non-breaking change labels Jun 25, 2026

github-actions Bot assigned Matt711 Jun 25, 2026

github-actions Bot added Python Affects Python cuDF API. cudf-polars Issues specific to cudf-polars labels Jun 25, 2026

github-project-automation Bot added this to cuDF Python Jun 25, 2026

GPUtester moved this to In Progress in cuDF Python Jun 25, 2026

Matt711 and others added 5 commits June 29, 2026 11:55

Merge branch 'main' into imp/polars/bump-polars-1.43

c82d12c

compat IR

5bc2314

only polars CI

e6887c6

Merge branch 'main' into imp/polars/bump-polars-1.43

cbaf750

Merge branch 'main' into imp/polars/bump-polars-1.43

f95b02b

Matt711 and others added 3 commits June 29, 2026 20:19

fix xpass'ing tests and implemented is_sorted boolean expr

5bb9a56

fallback for hive-partitioned scans, skip upstream polars tests

950fe2a

Merge branch 'main' into imp/polars/bump-polars-1.43

5f9aaf7

unpin polars version in pre-commit and a test for boolean is_sorted

93434d6

code coverage :(

1ad3e5f

Matt711 and others added 2 commits June 30, 2026 19:08

temp test all versions

6c041a4

Merge branch 'main' into imp/polars/bump-polars-1.43

585c4b0

Matt711 and others added 3 commits June 30, 2026 19:46

add dyn pred comment and revert polars only CI

7bb624f

xfail more upstream polars tests

0564da0

Merge branch 'main' into imp/polars/bump-polars-1.43

cb31390

Matt711 marked this pull request as ready for review June 30, 2026 20:34

Matt711 requested review from a team as code owners June 30, 2026 20:34

Matt711 requested a review from bdice June 30, 2026 20:34

coderabbitai Bot reviewed Jun 30, 2026

View reviewed changes

Comment thread python/cudf_polars/cudf_polars/testing/inject_gpu_engine.py Outdated

mroeschke reviewed Jun 30, 2026

View reviewed changes

Comment thread python/cudf_polars/cudf_polars/dsl/expressions/boolean.py Outdated

Comment thread python/cudf_polars/cudf_polars/testing/inject_gpu_engine.py Outdated

Matt711 and others added 2 commits June 30, 2026 23:49

fix incorrect DeprecationWarning failure in in xfail set, and use Col…

363ebe2

…umn.is_sorted

Merge branch 'main' into imp/polars/bump-polars-1.43

966a510

mroeschke approved these changes Jul 1, 2026

View reviewed changes

bdice approved these changes Jul 1, 2026

View reviewed changes

Merge branch 'main' into imp/polars/bump-polars-1.43

67ee448

update xfails

9741a1d

Matt711 requested a review from a team as a code owner July 2, 2026 17:47

Matt711 requested a review from msarahan July 2, 2026 17:47

Merge branch 'main' into imp/polars/bump-polars-1.43

7c60952

Matt711 commented Jul 2, 2026

View reviewed changes

bdice approved these changes Jul 2, 2026

View reviewed changes

Matt711 removed the request for review from msarahan July 2, 2026 17:59

skip tests

909017a

rapids-bot Bot merged commit 08f48c2 into NVIDIA:main Jul 2, 2026
138 checks passed

github-project-automation Bot moved this from In Progress to Done in cuDF Python Jul 2, 2026

Uh oh!

Conversation

Matt711 commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

copy-pr-bot Bot commented Jun 25, 2026

Uh oh!

Matt711 commented Jun 25, 2026

Uh oh!

Matt711 commented Jun 29, 2026

Uh oh!

Matt711 commented Jun 29, 2026

Uh oh!

Matt711 commented Jun 29, 2026

Uh oh!

Matt711 commented Jun 30, 2026

Uh oh!

Matt711 commented Jun 30, 2026

Uh oh!

coderabbitai Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Summary by CodeRabbit

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mroeschke left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Matt711 commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Matt711 commented Jul 1, 2026

Uh oh!

Matt711 Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

Matt711 commented Jul 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Matt711 commented Jun 25, 2026 •

edited

Loading

coderabbitai Bot commented Jun 30, 2026 •

edited

Loading

Matt711 commented Jun 30, 2026 •

edited

Loading