Skip to content

fix(cudf.pandas): remove stale xfail entries from pandas-testing-plugin - #22904

Merged
rapids-bot[bot] merged 12 commits into
NVIDIA:mainfrom
vyasr:stale-xfail-cleanup
Jun 23, 2026
Merged

fix(cudf.pandas): remove stale xfail entries from pandas-testing-plugin#22904
rapids-bot[bot] merged 12 commits into
NVIDIA:mainfrom
vyasr:stale-xfail-cleanup

Conversation

@vyasr

@vyasr vyasr commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

This PR aims to ensure that local runs of the pandas test workflow in a conda-based environment also match CI runs. That is important to reliably run agentic loops to try to fix pandas tests locally.

Remove 99 stale entries from NODEIDS_THAT_FAIL and fix the openpyxl node-ID mismatch in the pandas-testing-plugin:

Removed xfails (99 entries):

  • 97 tests/io/test_sql.py entries — these tests now pass because the required I/O packages (odfpy, pytables, python-calamine, pyxlsb, xlrd) are added as test dependencies
  • 1 test_range_difference — now passes
  • 1 test_dow_parametric — now passes

Fixed node-ID mismatch (net +1 entry):

  • Replaced the removed test_engine_kwargs_append_data_only (un-parametrized) with the two parametrized variants [True-0] and [False-=1+1] that actually exist in the pandas 3.0 test suite

Moved to NODEIDS_THAT_FLAKY_XFAIL (3 entries):

  • test_numpy_ufuncs_basic[complex64-log1p] — GPU-architecture-dependent xfail/xpass behavior
  • test_auto_detect — flaky under xdist due to terminal-width-dependent repr truncation
  • test_construct_dask_float_array_int_dtype_match_ndarray — flaky under xdist due to cross-test state interaction with dask
  • test_reindexing_with_float64_NA_log — moved from NODEIDS_THAT_FAIL to NODEIDS_THAT_FLAKY_XFAIL

Test environment changes:

  • Pin matplotlib<3.11 until pandas 3.0.4 releases with compat fix (pandas-dev/pandas#65742)
  • Add missing I/O test dependencies: odfpy, pytables, python-calamine, pyxlsb, xlrd

CI fix:

  • Add pre-commit clean in ci/check_style.sh to avoid stale pre-commit cache issues

@copy-pr-bot

copy-pr-bot Bot commented Jun 16, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@vyasr

vyasr commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test

@github-actions github-actions Bot added Python Affects Python cuDF API. cudf.pandas Issues specific to cudf.pandas labels Jun 16, 2026
@GPUtester GPUtester moved this to In Progress in cuDF Python Jun 16, 2026

@mroeschke mroeschke left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be useful to add some runtime validation to pytest_collection_modifyitems to catch these stale tests.

e.g. pytest_collection_modifyitems can pop entries from NODEIDS_THAT_FAIL and NODEIDS_TO_SKIP in its loop, and if either are non-empty at the end of the loop then we still have stale entries.

@vyasr
vyasr force-pushed the stale-xfail-cleanup branch from dfc32d2 to 63d4b65 Compare June 17, 2026 01:53
@vyasr

vyasr commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test

@vyasr vyasr added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Jun 17, 2026
@vyasr
vyasr force-pushed the stale-xfail-cleanup branch from 63d4b65 to c37194a Compare June 18, 2026 15:12
@vyasr

vyasr commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test

2 similar comments
@vyasr

vyasr commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test

@vyasr

vyasr commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test

@vyasr
vyasr force-pushed the stale-xfail-cleanup branch from 89a9b2f to dfe0b4b Compare June 21, 2026 23:16
@vyasr

vyasr commented Jun 21, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test

Pin matplotlib<3.11 in the pandas test dependency group until pandas
3.0.4 releases with matplotlib 3.11 compat fix (pandas-dev/pandas#65742).

Move three local-only non-regression tests to NODEIDS_TO_SKIP for
local/CI parity:
- complex64-log1p: GPU-architecture-dependent xfail/xpass behavior
- test_auto_detect: flaky under xdist due to terminal-width repr
- test_construct_dask_float_array_int_dtype_match_ndarray: flaky under
  xdist due to cross-test state interaction with dask
@vyasr
vyasr force-pushed the stale-xfail-cleanup branch from dfe0b4b to a67b4a1 Compare June 21, 2026 23:53
@vyasr

vyasr commented Jun 21, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test

…D mismatch

Remove 99 xfail entries that are deselected by the test runner's marker
expression (-m "not slow and not single_cpu and not db and not network"):
- 97 tests/io/test_sql.py entries (module-level single_cpu mark)
- 1 test_range_difference (slow mark)
- 1 test_dow_parametric (slow mark)

Fix 1 node-ID mismatch: test_engine_kwargs_append_data_only was listed
without parametrization but the test is parametrized as [True-0] and
[False-=1+1]. Split into the two correct entries.
@vyasr

vyasr commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test

@vyasr
vyasr marked this pull request as ready for review June 23, 2026 01:16
@vyasr
vyasr requested review from a team as code owners June 23, 2026 01:16
@vyasr
vyasr requested a review from a team as a code owner June 23, 2026 17:14
Comment thread ci/check_style.sh
vyasr added 2 commits June 23, 2026 13:27
# Conflicts:
#	ci/check_style.sh
#	python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
@vyasr

vyasr commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

/merge

@rapids-bot
rapids-bot Bot merged commit ab94837 into NVIDIA:main Jun 23, 2026
137 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in cuDF Python Jun 23, 2026
@vyasr
vyasr deleted the stale-xfail-cleanup branch June 23, 2026 21:54
mhaseeb123 pushed a commit to mhaseeb123/cudf that referenced this pull request Jun 24, 2026
…in (NVIDIA#22904)

This PR aims to ensure that local runs of the pandas test workflow in a conda-based environment also match CI runs. That is important to reliably run agentic loops to try to fix pandas tests locally.

Remove 99 stale entries from `NODEIDS_THAT_FAIL` and fix the openpyxl node-ID mismatch in the pandas-testing-plugin:

**Removed xfails (99 entries):**
- 97 `tests/io/test_sql.py` entries — these tests now pass because the required I/O packages (`odfpy`, `pytables`, `python-calamine`, `pyxlsb`, `xlrd`) are added as test dependencies
- 1 `test_range_difference` — now passes
- 1 `test_dow_parametric` — now passes

**Fixed node-ID mismatch (net +1 entry):**
- Replaced the removed `test_engine_kwargs_append_data_only` (un-parametrized) with the two parametrized variants `[True-0]` and `[False-=1+1]` that actually exist in the pandas 3.0 test suite

**Moved to `NODEIDS_THAT_FLAKY_XFAIL` (3 entries):**
- `test_numpy_ufuncs_basic[complex64-log1p]` — GPU-architecture-dependent xfail/xpass behavior
- `test_auto_detect` — flaky under xdist due to terminal-width-dependent repr truncation
- `test_construct_dask_float_array_int_dtype_match_ndarray` — flaky under xdist due to cross-test state interaction with dask
- `test_reindexing_with_float64_NA_log` — moved from `NODEIDS_THAT_FAIL` to `NODEIDS_THAT_FLAKY_XFAIL`

**Test environment changes:**
- Pin `matplotlib<3.11` until pandas 3.0.4 releases with compat fix ([pandas-dev/pandas#65742](pandas-dev/pandas#65742))
- Add missing I/O test dependencies: `odfpy`, `pytables`, `python-calamine`, `pyxlsb`, `xlrd`

**CI fix:**
- Add `pre-commit clean` in `ci/check_style.sh` to avoid stale pre-commit cache issues

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - Matthew Roeschke (https://github.com/mroeschke)
  - GALI PREM SAGAR (https://github.com/galipremsagar)
  - Bradley Dice (https://github.com/bdice)

URL: NVIDIA#22904
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cudf.pandas Issues specific to cudf.pandas improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants