Skip to content

Mark a few cuml-accel upstream tests as flaky - #7967

Merged
rapids-bot[bot] merged 1 commit into
NVIDIA:mainfrom
jcrist:flaky-time-based-tests
Apr 9, 2026
Merged

Mark a few cuml-accel upstream tests as flaky#7967
rapids-bot[bot] merged 1 commit into
NVIDIA:mainfrom
jcrist:flaky-time-based-tests

Conversation

@jcrist

@jcrist jcrist commented Apr 9, 2026

Copy link
Copy Markdown
Member

These tests match based on time in the logs, and can sometimes fail if an operation takes longer (possibly due to CI contention issues).

Fixes #7962

These tests match based on time in the logs, and can sometimes fail if
an operation takes longer (possibly due to CI contention issues).
@jcrist
jcrist requested a review from a team as a code owner April 9, 2026 01:45
@jcrist
jcrist requested a review from viclafargue April 9, 2026 01:45
@github-actions github-actions Bot added the Cython / Python Cython or Python issue label Apr 9, 2026
- "sklearn.model_selection.tests.test_validation::test_cross_val_predict[coo_matrix]"
- "sklearn.model_selection.tests.test_validation::test_fit_and_score_verbosity[False-scorer2-10-split_prg2-cdt_prg2-\\\\[CV 2/3; 1/1\\\\] END ....... sc1: \\\\(test=3.421\\\\) sc2: \\\\(test=3.421\\\\) total time= 0.\\\\ds]"
- "sklearn.model_selection.tests.test_validation::test_fit_and_score_verbosity[False-three_params_scorer-2-split_prg0-cdt_prg0-\\\\[CV\\\\] END .................................................... total time= 0.\\\\ds]"
- "sklearn.model_selection.tests.test_validation::test_fit_and_score_verbosity[True-scorer1-3-split_prg1-cdt_prg1-\\\\[CV 2/3\\\\] END sc1: \\\\(train=3.421, test=3.421\\\\) sc2: \\\\(train=3.421, test=3.421\\\\) total time= 0.\\\\ds]"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've seen at least one of these fail locally before. That and the fact that test_random_search_cv_results_multimetric was already marked as flaky makes me think this is less of a regression and more of us finally catching (and noting!) a failure for these.

@jcrist jcrist added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Apr 9, 2026
@coderabbitai

coderabbitai Bot commented Apr 9, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d115a613-7aa7-4139-8398-0f3d21c40941

📥 Commits

Reviewing files that changed from the base of the PR and between de94966 and b0e9fda.

📒 Files selected for processing (1)
  • python/cuml/cuml_accel_tests/upstream/scikit-learn/xfail-list.yaml

📝 Walkthrough

Summary by CodeRabbit

  • Chores
    • Updated test configuration with additional expected test behavior entries to reflect scikit-learn compatibility handling.

Walkthrough

The PR extends the xfail list configuration to mark specific flaky scikit-learn tests as expected failures. Four new test entries are added for timing-sensitive tests that fail intermittently in slow CI environments.

Changes

Cohort / File(s) Summary
xfail-list configuration
python/cuml/cuml_accel_tests/upstream/scikit-learn/xfail-list.yaml
Added four new xfail entries: one for test_grid_search_cv_results_multimetric and three for test_fit_and_score_verbosity test variants with parameterized selectors to handle flaky timing-related assertion failures.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related issues

  • #7962: This PR directly addresses the flaky test failures described in the issue by marking test_grid_search_cv_results_multimetric and test_fit_and_score_verbosity as expected failures due to strict timing assertions in slow CI environments.

Suggested labels

Cython / Python, non-breaking

Suggested reviewers

  • viclafargue
  • csadorf
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: marking upstream tests as flaky due to timing issues.
Description check ✅ Passed The description directly relates to the changeset, explaining that tests are flaky due to timing assumptions and referencing issue #7962.
Linked Issues check ✅ Passed The PR successfully addresses the objective of marking flaky timing-sensitive tests by adding them to the xfail list, directly resolving the issues documented in #7962.
Out of Scope Changes check ✅ Passed All changes are in-scope: the xfail-list.yaml modification adds only the flaky tests mentioned in issue #7962 without extraneous modifications.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@jcrist

jcrist commented Apr 9, 2026

Copy link
Copy Markdown
Member Author

/merge

@rapids-bot
rapids-bot Bot merged commit 4091267 into NVIDIA:main Apr 9, 2026
96 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Cython / Python Cython or Python issue improvement Improvement / enhancement to an existing function non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI] Flaky tests due to slow CI run

4 participants