Skip to content

Enable cudf.pandas test_numpy_ufuncs_basic via tolerant Index comparison - #22924

Merged
rapids-bot[bot] merged 6 commits into
NVIDIA:mainfrom
galipremsagar:tolerance_fix
Jun 23, 2026
Merged

Enable cudf.pandas test_numpy_ufuncs_basic via tolerant Index comparison#22924
rapids-bot[bot] merged 6 commits into
NVIDIA:mainfrom
galipremsagar:tolerance_fix

Conversation

@galipremsagar

@galipremsagar galipremsagar commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Description

The cudf.pandas pandas-tests plugin xfailed 77 tests/indexes/test_numpy_compat.py::test_numpy_ufuncs_basic[...] parametrizations. 75 of them were not real failures: the GPU results are numerically correct and only differ from NumPy's CPU results by ~1 ULP for transcendental ufuncs (sin, cos, tan, exp, log, sinh, ...). They were xfailed only because pandas._testing.assert_index_equal compares Index values exactly (check_exact=True) by default, so the last-bit drift trips the assertion even though the values agree to rtol=1e-5.

This PR lets those tests run and pass instead of being xfailed:

  • Adds a node-id-scoped, autouse fixture relax_exact_index_compare that, only for node ids matching TOLERANT_INDEX_COMPARE_SUBSTRINGS (currently test_numpy_ufuncs_basic), patches tm.assert_index_equal to inject check_exact=False (NumPy/pandas default rtol/atol) and restores it afterward. It reuses the existing replace_kwargs helper and is scoped by node id, so no other test's comparison is affected.
  • Drops the 75 now-passing entries from NODEIDS_THAT_FAIL.

Two entries are intentionally kept xfailed because they are not a tolerance issue: complex64-log2 and complex128-log2. There, log2(0+0j) yields -inf+nanj on the GPU (cupy) vs -inf+0j on the CPU (NumPy) — a structural nan-vs-0 mismatch in the imaginary part that no rtol/atol can bridge. Their xfail reason is updated to say so.

Safety / scope

  • The relaxation only loosens value comparison; dtype correctness is still enforced — test_numpy_ufuncs_basic makes its own explicit assert result.dtype == ... checks, which run independently of assert_index_equal.
  • The fixture is gated on the test node id, so the rest of the pandas-test suite continues to compare exactly.

Verification

Running test_numpy_ufuncs_basic with the updated plugin: 746 passed, 2 xfailed, 0 failures, 0 XPASS. The tolerance margin is comfortable (~100× for float32 ULP drift, far larger for complex128), so it should hold across CI hardware.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@copy-pr-bot

copy-pr-bot Bot commented Jun 18, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions github-actions Bot added Python Affects Python cuDF API. cudf.pandas Issues specific to cudf.pandas labels Jun 18, 2026
@GPUtester GPUtester moved this to In Progress in cuDF Python Jun 18, 2026
@galipremsagar galipremsagar changed the title fix Enable cudf.pandas test_numpy_ufuncs_basic via tolerant Index comparison Jun 18, 2026
@galipremsagar galipremsagar added 3 - Ready for Review Ready for review by team non-breaking Non-breaking change bug Something isn't working labels Jun 18, 2026
@galipremsagar
galipremsagar marked this pull request as ready for review June 18, 2026 20:34
@galipremsagar
galipremsagar requested a review from a team as a code owner June 18, 2026 20:34
@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 23eb506f-764d-4129-b61f-cd46f1ad28ae

📥 Commits

Reviewing files that changed from the base of the PR and between 974d4b0 and cbcc9ce.

📒 Files selected for processing (1)
  • python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py

📝 Walkthrough

Summary by CodeRabbit

  • Chores
    • Updated the testing framework to relax exact index equality checks for a targeted set of test cases, improving reliability for otherwise rigid comparisons.
    • Refined test discrepancy/xfail explanations for log2 scenarios, clarifying that mismatches come from structural NaN vs 0 differences rather than tolerance drift.
    • Removed outdated flaky or version-sensitive dispatch reason entries for nullable NumPy ufunc test cases.

Walkthrough

The pandas testing plugin gains a NODEIDS_TOLERANT_INDEX_COMPARE constant and marker registration to apply a tolerant_index_compare pytest marker to matching test node ids. An autouse fixture checks for this marker and temporarily monkeypatches pandas._testing.tm.assert_index_equal with check_exact=False and rtol=1e-6 for marked tests. Two log2-on-complex-input xfail reason strings are updated with explicit GPU-vs-CPU NaN-vs-0 descriptions, and a block of stale flaky nullable-ufunc entries is removed.

Changes

pandas testing plugin – tolerance fixture and xfail mapping cleanup

Layer / File(s) Summary
Marker registration and index-comparison fixture
python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
Adds NODEIDS_TOLERANT_INDEX_COMPARE constant and registers tolerant_index_compare marker via pytest_configure. pytest_collection_modifyitems applies the marker to items whose nodeid contains configured substrings. An autouse relax_exact_index_compare fixture monkeypatches pandas._testing.tm.assert_index_equal with check_exact=False and rtol=1e-6 for marked tests, restoring the original in a finally block.
xfail reason mapping cleanup
python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
Updates two test_numpy_ufuncs_basic log2-on-complex-input xfail entries with explicit GPU-vs-CPU NaN-vs-0 mismatch descriptions. Removes a block of stale "Flaky/version-sensitive cudf.pandas dispatch" reason entries for nullable ufunc parameterizations in test_numpy_compat.py.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • rapidsai/cudf#22643: Both PRs modify python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py, specifically the pytest_collection_modifyitems hook for test marker and configuration logic.

Suggested labels

improvement

Suggested reviewers

  • brandon-b-miller
  • TomAugspurger
  • Matt711
  • bdice
  • vyasr
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: enabling cudf.pandas tests by introducing tolerant index comparison.
Description check ✅ Passed The description is comprehensive and clearly related to the changeset, explaining the motivation, implementation, and verification of the tolerance-based index comparison feature.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

Comment thread python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py Outdated
Comment thread python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py Outdated
Comment thread python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py Outdated
@galipremsagar
galipremsagar requested a review from Matt711 June 22, 2026 14:50
Comment thread python/cudf/cudf/pandas/scripts/pandas-testing-plugin.py
@galipremsagar

Copy link
Copy Markdown
Contributor Author

/merge

@rapids-bot
rapids-bot Bot merged commit 9fcce1a into NVIDIA:main Jun 23, 2026
125 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in cuDF Python Jun 23, 2026
mhaseeb123 pushed a commit to mhaseeb123/cudf that referenced this pull request Jun 24, 2026
…rison (NVIDIA#22924)

The cudf.pandas `pandas-tests` plugin xfailed 77 `tests/indexes/test_numpy_compat.py::test_numpy_ufuncs_basic[...]` parametrizations. 75 of them were not real failures: the GPU results are numerically correct and only differ from NumPy's CPU results by ~1 ULP for transcendental ufuncs (`sin`, `cos`, `tan`, `exp`, `log`, `sinh`, ...). They were xfailed only because `pandas._testing.assert_index_equal` compares `Index` values **exactly** (`check_exact=True`) by default, so the last-bit drift trips the assertion even though the values agree to `rtol=1e-5`.

This PR lets those tests run and pass instead of being xfailed:

- Adds a node-id-scoped, autouse fixture `relax_exact_index_compare` that, only for node ids matching `TOLERANT_INDEX_COMPARE_SUBSTRINGS` (currently `test_numpy_ufuncs_basic`), patches `tm.assert_index_equal` to inject `check_exact=False` (NumPy/pandas default `rtol`/`atol`) and restores it afterward. It reuses the existing `replace_kwargs` helper and is scoped by node id, so no other test's comparison is affected.
- Drops the 75 now-passing entries from `NODEIDS_THAT_FAIL`.

Two entries are intentionally **kept** xfailed because they are *not* a tolerance issue: `complex64-log2` and `complex128-log2`. There, `log2(0+0j)` yields `-inf+nanj` on the GPU (cupy) vs `-inf+0j` on the CPU (NumPy) — a structural `nan`-vs-`0` mismatch in the imaginary part that no `rtol`/`atol` can bridge. Their xfail reason is updated to say so.

### Safety / scope

- The relaxation only loosens **value** comparison; dtype correctness is still enforced — `test_numpy_ufuncs_basic` makes its own explicit `assert result.dtype == ...` checks, which run independently of `assert_index_equal`.
- The fixture is gated on the test node id, so the rest of the pandas-test suite continues to compare exactly.

### Verification

Running `test_numpy_ufuncs_basic` with the updated plugin: **746 passed, 2 xfailed, 0 failures, 0 XPASS**. The tolerance margin is comfortable (~100× for float32 ULP drift, far larger for complex128), so it should hold across CI hardware.

Authors:
  - GALI PREM SAGAR (https://github.com/galipremsagar)

Approvers:
  - Matthew Murray (https://github.com/Matt711)

URL: NVIDIA#22924
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3 - Ready for Review Ready for review by team bug Something isn't working cudf.pandas Issues specific to cudf.pandas non-breaking Non-breaking change Python Affects Python cuDF API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants