Skip to content

Convert conftest-patch.py to a pytest plugin for pandas testing - #22643

Merged
rapids-bot[bot] merged 2 commits into
NVIDIA:mainfrom
mroeschke:cudf_pandas/ref/pandas_testing_plugin
May 26, 2026
Merged

Convert conftest-patch.py to a pytest plugin for pandas testing#22643
rapids-bot[bot] merged 2 commits into
NVIDIA:mainfrom
mroeschke:cudf_pandas/ref/pandas_testing_plugin

Conversation

@mroeschke

Copy link
Copy Markdown
Contributor

Description

This PR converts and renames conftest-patch.py to apandas-testing-plugin.py pytest plugin file that can be used as pytest -p pandas-testing-plugin to patch pandas testing utilities and dynamically add our xfail or skip markers instead of modifying the existing pandas' conftest.py.

Hoping to reduce the modifications we make to the cloned repo in a follow up PR

Additionally moves some flags set in CI scripts to the script run locally for better CI/local alignment

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@mroeschke mroeschke self-assigned this May 22, 2026
@mroeschke mroeschke added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels May 22, 2026
@copy-pr-bot

copy-pr-bot Bot commented May 22, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions github-actions Bot added Python Affects Python cuDF API. cudf.pandas Issues specific to cudf.pandas labels May 22, 2026
@GPUtester GPUtester moved this to In Progress in cuDF Python May 22, 2026
@mroeschke

Copy link
Copy Markdown
Contributor Author

/ok to test 429b4ad

@mroeschke

Copy link
Copy Markdown
Contributor Author

/ok to test a549ef8

@mroeschke
mroeschke marked this pull request as ready for review May 23, 2026 02:58
@mroeschke
mroeschke requested review from a team as code owners May 23, 2026 02:58
@coderabbitai

coderabbitai Bot commented May 23, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Summary by CodeRabbit

  • Tests

    • Improved Pandas test infrastructure with restructured test plugin and assertion handling.
    • Updated test invocation to register custom testing plugins for enhanced compatibility checks.
  • Chores

    • Updated linting configuration to remove redundant ignore patterns.
    • Reorganized test script imports and dependencies for better maintainability.

Walkthrough

This PR refactors pandas test warning suppression by moving helper logic from the pytest plugin into a wrapper module, updating the plugin to properly register as a pytest hook, and configuring test runners to load the new plugin directly instead of patching conftest files.

Changes

Pandas Test Warning Suppression

Layer / File(s) Summary
Warning suppression helpers in wrapper module
python/cudf/cudf/pandas/_wrappers/pandas.py
Adds contextlib import and defines null_assert_produces_warning and null_raises_chained_assignment_error context managers that redirect pandas testing assertions to suppress warnings and chained-assignment errors.
Plugin refactoring and hook registration
python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
Removes the old null_assert_warnings context manager from the plugin, cleans up unused imports (contextlib, os), adds explicit @pytest.hookimpl(trylast=True) decorator to pytest_collection_modifyitems, and removes sys.path.append statement.
Test runner and configuration updates
python/cudf/cudf/pandas/scripts/run-pandas-tests.sh, ci/cudf_pandas_scripts/pandas-tests/run.sh, pyproject.toml
Registers the pytest plugin via -p cudf.pandas.scripts.pandas-testing-plugin, removes the conftest patching approach, expands marker filters to exclude network tests, removes stale comment guidance, and deletes the Ruff F821 ignore for conftest-patch.py.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes


Suggested reviewers

  • pentschev
  • TomAugspurger
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and accurately describes the main change: converting conftest-patch.py to a pytest plugin for pandas testing, which aligns with the substantial refactoring in the changeset.
Description check ✅ Passed The description is directly related to the changeset, explaining the conversion of conftest-patch.py to a pytest plugin and the movement of CI flags, matching the actual code changes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/cudf/cudf/pandas/_wrappers/pandas.py`:
- Around line 1512-1536: Add unit tests and a benchmark for the new proxy helper
functions: null_assert_produces_warning, _register_assert_produces_warning,
null_raises_chained_assignment_error, and
_register_raises_chained_assignment_error. Create pytest cases that verify each
helper returns a contextmanager, that using the context manager
suppresses/handles pandas warnings and chained-assignment errors as intended
(exercise both direct calls to null_* helpers and the registered proxy wrappers
_register_*), and assert proper enter/exit behavior; also add a lightweight perf
benchmark (pytest-benchmark or project benchmark style) that measures
entering/exiting these context managers to satisfy repo policy. Ensure tests
live under the repo's test pattern and include necessary imports/mocks for
pd._testing so the proxy registration is exercised.
- Around line 1512-1520: The current null_assert_produces_warning context
manager yields an empty list and doesn't capture warnings, so under
filterwarnings("error")/ -Werror warnings still raise; modify
null_assert_produces_warning to use warnings.catch_warnings(record=True) and set
a permissive filter (e.g., warnings.simplefilter("always") or "default") inside
the context so warnings are recorded rather than turned into errors, then yield
the recorded list and restore warning state on exit; update the function
null_assert_produces_warning to import and use the warnings module and return
the captured warnings list similarly to pandas.assert_produces_warning.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 231faff8-bf1e-48df-bd51-acdacba82586

📥 Commits

Reviewing files that changed from the base of the PR and between 3cfe15b and a549ef8.

📒 Files selected for processing (5)
  • ci/cudf_pandas_scripts/pandas-tests/run.sh
  • pyproject.toml
  • python/cudf/cudf/pandas/_wrappers/pandas.py
  • python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
  • python/cudf/cudf/pandas/scripts/run-pandas-tests.sh
💤 Files with no reviewable changes (2)
  • ci/cudf_pandas_scripts/pandas-tests/run.sh
  • pyproject.toml

Comment thread python/cudf/cudf/pandas/_wrappers/pandas.py
Comment thread python/cudf/cudf/pandas/_wrappers/pandas.py
@mroeschke
mroeschke requested a review from vyasr May 26, 2026 17:46
def null_assert_produces_warning(*args, **kwargs):
# We do not want pandas unit tests to fail because
# assert_produces_warning doesn't see a warning.
# No an explicit public API

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this comment mean?

@mroeschke mroeschke May 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose I was trying to convey "this is a proxy that doesn't faithfully fall back to pandas because we need it to be a no-op when we run pandas tests". I can tune this comment in a follow up PR or a next commit if needed.

@mroeschke

Copy link
Copy Markdown
Contributor Author

/merge

@rapids-bot
rapids-bot Bot merged commit f2a06c6 into NVIDIA:main May 26, 2026
219 of 221 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in cuDF Python May 26, 2026
@mroeschke
mroeschke deleted the cudf_pandas/ref/pandas_testing_plugin branch May 26, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cudf.pandas Issues specific to cudf.pandas improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants