Skip to content

Improve timeout verbosity and graceful shutdown for pytest CI jobs - #7979

Merged
rapids-bot[bot] merged 2 commits into
NVIDIA:mainfrom
csadorf:ci/improve-timeout-handling
Apr 22, 2026
Merged

Improve timeout verbosity and graceful shutdown for pytest CI jobs#7979
rapids-bot[bot] merged 2 commits into
NVIDIA:mainfrom
csadorf:ci/improve-timeout-handling

Conversation

@csadorf

@csadorf csadorf commented Apr 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds -v to all timeout invocations that were missing it, so cancellations are reported to the CI log.
  • Adds --signal=SIGINT --kill-after=60s to every timeout wrapping a pytest invocation, so pytest receives SIGINT on timeout (prints a partial test summary) and is hard-killed 60 s later if it has not exited.
  • Non-pytest timeout calls (run_ctests.sh, BERTopic smoke test) are left unchanged (already had -v).

Eight files changed, all in ci/: test_python_singlegpu.sh, test_python_integration.sh, test_python_dask.sh, test_python_scikit_learn_tests.sh, test_python_sklearn_examples.sh, test_python_cuml_accel_upstream.sh, test_wheel.sh, test_wheel_dask.sh.

- increase verbosity
- attempt graceful shutdown of pytest invocations
@csadorf
csadorf requested a review from a team as a code owner April 21, 2026 19:06
@csadorf
csadorf requested a review from msarahan April 21, 2026 19:06
@github-actions github-actions Bot added the ci label Apr 21, 2026
@csadorf csadorf added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Apr 21, 2026
@coderabbitai

coderabbitai Bot commented Apr 21, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 19b20ec2-da81-46b0-b323-17639e304e1b

📥 Commits

Reviewing files that changed from the base of the PR and between 9f83155 and 82b4517.

📒 Files selected for processing (8)
  • ci/test_python_cuml_accel_upstream.sh
  • ci/test_python_dask.sh
  • ci/test_python_integration.sh
  • ci/test_python_scikit_learn_tests.sh
  • ci/test_python_singlegpu.sh
  • ci/test_python_sklearn_examples.sh
  • ci/test_wheel.sh
  • ci/test_wheel_dask.sh

📝 Walkthrough

Summary by CodeRabbit

Release Notes

  • Chores
    • Updated test execution timeouts across CI pipelines to use graceful interrupt signals with a 60-second grace period before forced termination, improving cleanup behavior
    • Enhanced timeout operation visibility with verbose logging for better test execution monitoring

Walkthrough

Eight CI shell scripts are updated to modify timeout command invocations, adding explicit signal handling (--signal=SIGINT) and post-interrupt termination behavior (--kill-after=60s) to improve process cleanup on test timeouts.

Changes

Cohort / File(s) Summary
Python test runners
ci/test_python_cuml_accel_upstream.sh, ci/test_python_dask.sh, ci/test_python_integration.sh, ci/test_python_scikit_learn_tests.sh, ci/test_python_singlegpu.sh, ci/test_python_sklearn_examples.sh
Updated timeout invocations to add --signal=SIGINT --kill-after=60s flags and verbose output (-v where applicable), changing how test processes are interrupted and terminated on timeout.
Wheel test runners
ci/test_wheel.sh, ci/test_wheel_dask.sh
Updated timeout invocations wrapping test scripts to include --signal=SIGINT --kill-after=60s flags, providing explicit interrupt signal and forced termination behavior.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

Suggested labels

ci, improvement, non-breaking

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: improving timeout verbosity and graceful shutdown for pytest CI jobs, which accurately reflects the core objective of adding -v flags and --signal=SIGINT --kill-after=60s to timeout invocations.
Description check ✅ Passed The description is directly related to the changeset, providing clear context on what was changed (8 CI files), why it was changed (better logging and graceful shutdown), and which timeout invocations were modified versus left unchanged.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@csadorf

csadorf commented Apr 22, 2026

Copy link
Copy Markdown
Contributor Author

/merge

@rapids-bot
rapids-bot Bot merged commit c7b104c into NVIDIA:main Apr 22, 2026
93 checks passed
@csadorf
csadorf deleted the ci/improve-timeout-handling branch April 22, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci improvement Improvement / enhancement to an existing function non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants