Handle sklearn example OpenML network failures

[csadorf]

Treat additional connection, HTTP, requests, socket, and SSL failures from sklearn example runs as network xfails, and scan stdout as well as stderr so OpenML fetch failures do not fail the examples job.

Closes #8204

[coderabbitai]

📝 Walkthrough

Summary by CodeRabbit

• Tests
  • Improved network-failure detection in test runs: output is now scanned more broadly (both stdout and stderr), causing network-related issues to emit a network-error warning and be marked as expected failures to reduce noise and improve failure classification.

Walkthrough

This PR expands the network-error substring list, adds a helper to find the first matching pattern in process output, and applies detection to stdout+stderr combined to emit ExampleNetworkError and xfail with the matched pattern.

Changes

Enhanced network error detection

Layer / File(s)	Summary
Network error detection refactoring python/cuml/cuml_accel_tests/upstream/scikit-learn/example_collector.py	_NETWORK_ERROR_PATTERNS extended; new _network_error_pattern(output) returns the first matching substring from process output; example execution now checks stderr+stdout and emits ExampleNetworkError and xfail("Network error: {pattern}") when matched.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• rapidsai/cuml#7960: Prior changes to the same example_collector.py test runner that introduced example collection and initial failure classification enhancements.

Suggested reviewers

• betatim

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title directly and concisely describes the main change: handling sklearn example OpenML network failures.
Description check	✅ Passed	The description is directly related to the changeset, explaining the additional network error patterns being added and the stdout/stderr scanning improvement.
Linked Issues check	✅ Passed	The PR implements the objective stated in issue #8204: treats additional network failures as xfails and scans both stdout and stderr to prevent OpenML fetch failures from failing the examples job.
Out of Scope Changes check	✅ Passed	All changes are scoped to the network error pattern detection in the example collector, directly addressing the linked issue objective.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/cuml/cuml_accel_tests/upstream/scikit-learn/example_collector.py`:
- Around line 115-126: The current failure path truncates and raises only
stderr, losing stdout diagnostics; modify the block in ExampleCollector (where
stderr/result and pattern = _network_error_pattern(result.stderr +
result.stdout) are used) to preserve stdout in the raised ExampleFailed message:
construct a combined diagnostics string that includes both stdout and stderr
(apply the same 4000-char truncation to the tail of the combined text or to
stderr and stdout individually as desired), and raise ExampleFailed with that
combined text while keeping the existing network-error xfail branch that raises
pytest.xfail(…) for ExampleNetworkError.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 32b108a0-a1d0-42ef-b3e4-6152bde1d7e4

📥 Commits

Reviewing files that changed from the base of the PR and between beba7db and 9a059d1.

📒 Files selected for processing (1)

• python/cuml/cuml_accel_tests/upstream/scikit-learn/example_collector.py

[jcrist]

[coderabbitai]

♻️ Duplicate comments (1)

python/cuml/cuml_accel_tests/upstream/scikit-learn/example_collector.py (1)

124-126: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Preserve stdout in the non-xfail failure path.

Line 116 correctly inspects both streams for network errors, but lines 124-126 still truncate and raise only stderr. When failures appear in stdout but aren't network-related, their diagnostics are lost.

Suggested fix

         if result.returncode != 0:
-            stderr = result.stderr
-            pattern = _network_error_pattern(result.stderr + result.stdout)
+            output = result.stderr + result.stdout
+            pattern = _network_error_pattern(output)
             if pattern:
                 warnings.warn(
                     f"Example {self.path.name} failed due to network error"
                     f" ({pattern})",
                     ExampleNetworkError,
                 )
                 pytest.xfail(reason=f"Network error: {pattern}")
-            if len(stderr) > 4000:
-                stderr = "...\n" + stderr[-4000:]
-            raise ExampleFailed(stderr)
+            if len(output) > 4000:
+                output = "...\n" + output[-4000:]
+            raise ExampleFailed(output)

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/cuml/cuml_accel_tests/upstream/scikit-learn/example_collector.py`
around lines 124 - 126, When raising ExampleFailed currently only stderr is
included, lose diagnostics from stdout; update the failure path that raises
ExampleFailed to include stdout as well (e.g., combine truncated stdout and
stderr into the error message) using the same truncation logic you apply to
stderr (keep last 4000 chars, prefix with "...\n" if truncated). Modify the
block around the raise so ExampleFailed receives both streams (reference
symbols: ExampleFailed, stdout, stderr) while leaving the earlier network-error
inspection intact.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@python/cuml/cuml_accel_tests/upstream/scikit-learn/example_collector.py`:
- Around line 124-126: When raising ExampleFailed currently only stderr is
included, lose diagnostics from stdout; update the failure path that raises
ExampleFailed to include stdout as well (e.g., combine truncated stdout and
stderr into the error message) using the same truncation logic you apply to
stderr (keep last 4000 chars, prefix with "...\n" if truncated). Modify the
block around the raise so ExampleFailed receives both streams (reference
symbols: ExampleFailed, stdout, stderr) while leaving the earlier network-error
inspection intact.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d4a696b9-d944-4271-86de-01d3eb46b044

📥 Commits

Reviewing files that changed from the base of the PR and between 9a059d1 and 21055f6.

📒 Files selected for processing (1)

• python/cuml/cuml_accel_tests/upstream/scikit-learn/example_collector.py