Fix Series.isin dtype and value semantics for masked (nullable) dtypes - #23057
Merged
Conversation
github-actions
Bot
added
Python
galipremsagar
added
bug
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (3)
💤 Files with no reviewable changes (1)
✅ Files skipped from review due to trivial changes (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughUpdates Changesisin dtype handling fix
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@python/cudf/cudf/core/series.py`:
- Around line 3163-3168: The nullable-dtype path in `Series.isin` is iterating
over `values` twice in Python (`cleaned_values` and the later `any(...)`), which
can exhaust one-shot iterables and cause repeated GPU-to-host reads. Update the
`Series.isin` logic to normalize and inspect `values` once before building
`cleaned_values`, ideally by materializing or otherwise reusing a single pass
result, and then derive both the NA check and the filtered lookup list from that
same cached representation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 7657cffc-4425-4de0-825b-ad35066f3e9d
📒 Files selected for processing (4)
python/cudf/cudf/core/column/numerical.pypython/cudf/cudf/core/series.pypython/cudf/cudf/pandas/scripts/pandas-testing-plugin.pypython/cudf/cudf/tests/series/methods/test_isin.py
💤 Files with no reviewable changes (1)
- python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
Contributor
Author
|
/okay to test 6c4b51b |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
python/cudf/cudf/tests/series/methods/test_isin.py (1)
278-299: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winInconsistent dtype assertions across sub-blocks.
The Int64 block asserts
got.dtype == pd.BooleanDtype()(Line 287), but the boolean block (Lines 290-293) and Float64 block (Lines 295-298) omit the equivalent check ongot.dtype, only verifyinggsr.dtype. Since this test specifically exists to guard nullable-dtype isin behavior in pandas-compatible mode, all three cases should assert the output dtype consistently.🧪 Proposed fix to add missing dtype assertions
gsr = cudf.Series([True, pd.NA], dtype="boolean") got = gsr.isin([True]) assert gsr.dtype == pd.BooleanDtype() + assert got.dtype == pd.BooleanDtype() assert got.to_pandas().tolist() == [True, False] gsr = cudf.Series([1.5, 2.5], dtype="Float64") got = gsr.isin([1.5]) assert gsr.dtype == pd.Float64Dtype() + assert got.dtype == pd.BooleanDtype() assert got.to_pandas().tolist() == [True, False]As per coding guidelines,
python/**/test_*.pyshould "Ensure test files provide comprehensive edge case coverage."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@python/cudf/cudf/tests/series/methods/test_isin.py` around lines 278 - 299, The nullable-dtype isin test is inconsistent because only the Int64 case checks the result dtype, while the boolean and Float64 cases in test_isin_masked_pandas_compatible_mode omit the same assertion. Update that test so each sub-block verifies both the input Series dtype and the returned mask dtype, using the existing got variable and pd.BooleanDtype() consistently across all three cases.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@python/cudf/cudf/tests/series/methods/test_isin.py`:
- Around line 278-299: The nullable-dtype isin test is inconsistent because only
the Int64 case checks the result dtype, while the boolean and Float64 cases in
test_isin_masked_pandas_compatible_mode omit the same assertion. Update that
test so each sub-block verifies both the input Series dtype and the returned
mask dtype, using the existing got variable and pd.BooleanDtype() consistently
across all three cases.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 7d90a29e-628f-4a1c-99f4-5c6109f1b2d4
📒 Files selected for processing (3)
python/cudf/cudf/core/series.pypython/cudf/cudf/pandas/scripts/pandas-testing-plugin.pypython/cudf/cudf/tests/series/methods/test_isin.py
💤 Files with no reviewable changes (1)
- python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
🚧 Files skipped from review as they are similar to previous changes (1)
- python/cudf/cudf/core/series.py
Contributor
Author
|
/okay to test 230446e |
mroeschke
requested changes
Jul 2, 2026
Contributor
Author
|
/okay to test 8b22815 |
# Conflicts: # python/cudf/cudf/tests/series/methods/test_isin.py
Contributor
Author
|
/okay to test 4c80565 |
mroeschke
approved these changes
Jul 13, 2026
* Simplify the masked-isin dispatch to an isinstance check on np.dtype/pd.ArrowDtype (equivalent for NumericalColumn dtypes) and drop the now-unused import. * Inline the isin result column into Series._from_column.
Contributor
Author
|
/okay to test 9cf3a34 |
galipremsagar
added
5 - Ready to Merge
Contributor
Author
|
/merge |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes the
Series.isinfailures intests/series/methods/test_isin.pyundercudf.pandas(test_isin_masked_typesandtest_isin_large_series_and_pdNA, 12 cases). Two root causes, both fixed in cudf-classic:1. Result dtype for masked (nullable) inputs
Series.isinon a pandas masked dtype (Int64/Float64/boolean) returned a numpybool, but pandas returns a nullableboolean.Series.isinnow mirrors pandas'BaseMaskedArray.isinfor masked inputs:1),pd.NAitself is one ofvalues(a plainNaN/None/NaTdoes not match), andBooleanDtype.Non-masked dtypes (numpy, arrow, nullable-string, categorical) keep the numpy
boolresult, matching pandas.2. Boolean-vs-numeric comparison bailed out
A boolean column compared as all-
Falseagainst numeric needles becausecan_cast_safelyreports bool↔numeric as unsafe, soColumnBase.isinreturned an all-Falseresult.NumericalColumn._process_values_for_isinnow promotes the boolean side to the numeric dtype (a bool always fits), soTrue == 1compares by value like numpy/pandas. This also fixes plaincudf.Series([True, False]).isin([1])in cudf-classic.The now-passing entries are removed from the
cudf.pandasxfail list.Tests
Added cudf-classic coverage in
tests/series/methods/test_isin.py:test_isin_masked_types— mirrors pandas' cases acrossboolean/Int64/Float64, asserting theBooleanDtyperesult and NA semantics,test_isin_bool_against_numeric— boolean Series vs numericvalues,test_isin_non_masked_extension_returns_numpy_bool— arrow / categorical inputs yield numpybool.Checklist