Skip to content

Revert xfail_strict=false for pandas tests - #22913

Merged
rapids-bot[bot] merged 3 commits into
NVIDIA:mainfrom
vyasr:revert-xfail-strict-false
Jun 17, 2026
Merged

Revert xfail_strict=false for pandas tests#22913
rapids-bot[bot] merged 3 commits into
NVIDIA:mainfrom
vyasr:revert-xfail-strict-false

Conversation

@vyasr

@vyasr vyasr commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Reverts the temporary xfail_strict=false setting introduced in #22677. If flaky xpasses surface, we should add them to the skip list and investigate individually rather than globally disabling strict xfail. Leaving xfail_strict = false silently allows tests that should be failing to pass without anyone noticing, and then they could start failing again later. We want to make sure we're always trending upwards in our pass rate.

Closes #22681

Reverts the temporary xfail_strict=false setting introduced in NVIDIA#22677.
If flaky xpasses surface, we should add them to the skip list and
investigate individually rather than globally disabling strict xfail.

Closes NVIDIA#22681
@vyasr
vyasr requested a review from a team as a code owner June 17, 2026 01:01
@vyasr
vyasr requested review from mroeschke and rjzamora June 17, 2026 01:01
@github-actions github-actions Bot added Python Affects Python cuDF API. cudf.pandas Issues specific to cudf.pandas labels Jun 17, 2026
@GPUtester GPUtester moved this to In Progress in cuDF Python Jun 17, 2026
@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

In run-pandas-tests.sh, the generated pandas-tests/pyproject.toml pytest configuration is updated to set xfail_strict from false to true, causing unexpected test passes (xpasses) to be treated as failures. SPDX copyright header lines are added to both scripts. Simultaneously, pandas-testing-plugin.py receives comprehensive updates to its expected-failure mappings catalog, replacing generic TODO messages with specific failure reasons, adding GH issue references, and introducing new xfail/xpass/flaky classifications across categorical, groupby, index, series, extension array, and IO/plotting test categories.

Changes

pytest configuration and pandas test expected outcomes

Layer / File(s) Summary
Pytest configuration and copyright headers
python/cudf/cudf/pandas/scripts/run-pandas-tests.sh, python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
Adds SPDX copyright header lines to both files and sets xfail_strict = true in the generated pandas-tests/pyproject.toml, reverting the temporary false setting from #22677.
Categorical type and base conversion expectations
python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
Updates expected-failure mappings for categorical constructor, indexing, missing-case, and base type conversion scenarios (to_numpy, values_consistent).
GroupBy operation expectations
python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
Refines xfail mappings across groupby aggregate, all-methods, cumulative, mixed, and transform scenarios with specific failure annotations and TODO entries.
Index, period, and temporal type expectations
python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
Updates expected-failure mappings for IntervalIndex timezone conversions, PeriodIndex joins, and frame sorting/CSV export operations.
Series method expectations
python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
Adjusts expected-failure mappings for series sort-values and index-related operations with explicit assertion expectations.
Extension array and arithmetic expectations
python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
Adds and refines xfail mappings for Arrow extension arrays (arithmetic, scalar, accumulate, reduce with skew/kurtosis parameters), NumPy object-dtype variants, and arithmetic string tests with pandas-xfails-but-xpasses-with-cudf.pandas annotations.
Frame, query, finalize, IO, and plotting expectations
python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
Extends expected-failure mappings for extension array eval/finalize categorical accessor tests, flaky groupby/numba cases, JSON readlines Unicode handling, pyarrow parser EOF states, and matplotlib datetime64 plotting.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

  • rapidsai/cudf#22677: Directly modifies the same xfail_strict setting in the same file, switching it from true to false — this PR reverses that change.
  • rapidsai/cudf#22643: Updates the same pandas-testing-plugin.py expected-outcome mappings by changing how pandas test expectations and xfail outcomes are applied.
  • rapidsai/cudf#22883: Modifies pandas-testing-plugin.py's xfail mappings by removing stale expected-failure entries, addressing known test outcome cleanup.

Suggested reviewers

  • TomAugspurger
  • brandon-b-miller
  • bdice
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Revert xfail_strict=false for pandas tests' clearly and concisely describes the main change in the PR: reverting a temporary xfail_strict setting.
Linked Issues check ✅ Passed The PR addresses issue #22681 by reverting the temporary xfail_strict=false workaround and implementing the proper approach of investigating and handling flaky tests individually as proposed.
Out of Scope Changes check ✅ Passed All changes are in-scope: reverting xfail_strict to true in run-pandas-tests.sh and updating test expectation mappings in pandas-testing-plugin.py to handle individual flaky tests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description check ✅ Passed The pull request description clearly explains the revert of xfail_strict=false, the rationale for why it was problematic, and the preferred approach for handling flaky tests.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@vyasr vyasr added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Jun 17, 2026
- Remove 36 stale entries from NODEIDS_THAT_FAIL (tests now pass)
- Add 53 pandas-internally-xfailed tests to NODEIDS_TO_SKIP with
  descriptive reasons (these xpass under cudf.pandas)
- Skip flaky test_groupby_all_nan_groups_drop (intermittent assertion
  failure tracked in NVIDIA#22681)
@vyasr

vyasr commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

/merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cudf.pandas Issues specific to cudf.pandas improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

[BUG]: Flaky pandas-tests job

5 participants