Skip to content

Apply new input validation to metrics.trustworthiness - #8058

Merged
rapids-bot[bot] merged 5 commits into
NVIDIA:mainfrom
csadorf:issue-7998-apply-new-validation-to-metrics-trustworthiness
May 7, 2026
Merged

Apply new input validation to metrics.trustworthiness#8058
rapids-bot[bot] merged 5 commits into
NVIDIA:mainfrom
csadorf:issue-7998-apply-new-validation-to-metrics-trustworthiness

Conversation

@csadorf

@csadorf csadorf commented May 6, 2026

Copy link
Copy Markdown
Contributor

Applies new input validation system to metrics.trustworthiness.

Part of #7998

@csadorf
csadorf requested a review from a team as a code owner May 6, 2026 18:28
@csadorf
csadorf requested a review from jcrist May 6, 2026 18:28
@copy-pr-bot

copy-pr-bot Bot commented May 6, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the Cython / Python Cython or Python issue label May 6, 2026
@csadorf csadorf added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change and removed Cython / Python Cython or Python issue labels May 6, 2026
@coderabbitai

coderabbitai Bot commented May 6, 2026

Copy link
Copy Markdown

Review Change Stack
No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3e37655e-ae75-4dee-9da2-06f05661ac7d

📥 Commits

Reviewing files that changed from the base of the PR and between 1678d35 and 8c1b111.

📒 Files selected for processing (2)
  • python/cuml/cuml/metrics/trustworthiness.pyx
  • python/cuml/tests/test_trustworthiness.py

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes

    • Improved input validation and clearer error reporting for invalid sample counts, neighbor constraints, unsupported metrics, and dimensionality.
  • Changes

    • Default behavior now converts input dtypes automatically, reducing dtype-related errors for the trustworthiness metric.
  • Tests

    • Expanded coverage with targeted tests for invalid neighbors, mismatched sample counts, dtype conversion behavior, 1D inputs, and unknown metrics.

Walkthrough

The PR refactors the trustworthiness() function to replace input_to_cuml_array with check_array/check_consistent_length, extracts device pointers from validated arrays, changes convert_dtype default to True, adjusts neighbor validation to use validated n_samples, updates the backend call, and adds tests covering invalid n_neighbors, mismatched rows, dtype conversion, 1D inputs, and unknown metrics.

Changes

Trustworthiness Input Validation Refactor

Layer / File(s) Summary
Imports & Documentation
python/cuml/cuml/metrics/trustworthiness.pyx
Replace input_to_cuml_array import with check_array and check_consistent_length; update docstring to set convert_dtype=True default.
Input Validation & Pointer Setup
python/cuml/cuml/metrics/trustworthiness.pyx
Validate/convert X and X_embedded to contiguous np.float32, derive n_samples, n_features, n_components, enforce consistent sample counts, and extract device pointers d_X_ptr, d_X_embedded_ptr.
Neighbor Bounds & Backend Call
python/cuml/cuml/metrics/trustworthiness.pyx
Use validated n_samples for n_neighbors bounds check; call trustworthiness_score with extracted device pointers and adjusted dimensional args; keep handle.sync().
Tests / Validation Coverage
python/cuml/tests/test_trustworthiness.py
Replace generic invalid-input test with parameterized test_trustworthiness_invalid_n_neighbors; add tests for mismatched rows, convert_dtype=False rejecting float64, 1D input rejection, and unsupported metric handling.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

  • rapidsai/cuml#8057: Performs the same input_to_cuml_array → check_array refactor for a different metric (kl_divergence).
  • rapidsai/cuml#8043: Migrates array validation to check_array/check_consistent_length and updates pointer extraction patterns across metrics.
  • rapidsai/cuml#8059: Similar metrics migration applying check_array-based validation and .data.ptr extraction in related metric code.

Suggested labels

Cython / Python

Suggested reviewers

  • jcrist
  • betatim
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely describes the main change: applying new input validation to the metrics.trustworthiness function.
Description check ✅ Passed The description is directly related to the changeset, explaining that it applies the new input validation system to metrics.trustworthiness as part of issue #7998.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/cuml/cuml/metrics/trustworthiness.pyx`:
- Around line 83-104: The code currently calls check_array on X and X_embedded
before validating the requested metric; move or add an early validation of the
metric (e.g., in the trustworthiness function) to raise a clear ValueError for
unsupported metric values before performing any check_array conversions so the
heavy allocations for X_m and X_m2 (and their data.ptr conversions d_X_ptr /
d_X_embedded_ptr) are avoided on invalid input; ensure the validation references
the same metric parameter name used in the function signature and matches the
allowed metric set.
- Around line 105-106: Update the n_neighbors validation in the trustworthiness
implementation so it matches scikit-learn: in the trustworthiness function (in
python/cuml/cuml/metrics/trustworthiness.pyx) replace the current check that
compares n_neighbors to n_samples with a check that raises a ValueError when
n_neighbors >= n_samples / 2 (i.e., require n_neighbors < n_samples / 2); keep
the error message clear and reference n_samples and n_neighbors for context.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3273cf85-51fc-4220-9130-d5693a42f877

📥 Commits

Reviewing files that changed from the base of the PR and between b51c429 and 1678d35.

📒 Files selected for processing (2)
  • python/cuml/cuml/metrics/trustworthiness.pyx
  • python/cuml/tests/test_trustworthiness.py

Comment thread python/cuml/cuml/metrics/trustworthiness.pyx
Comment thread python/cuml/cuml/metrics/trustworthiness.pyx Outdated
csadorf added 5 commits May 7, 2026 17:09
Drive-by cleanups:
* Remove a duplicated 'n_neighbors > X.shape[0]' check.
* Validate that X and X_embedded have matching numbers of rows.
* Move the n_neighbors check after the input validation so it runs
  against the validated, coerced array shape.
The docstring stated default=False but the signature defaults to True.
Tighten the existing n_neighbors check to match on the error message,
and add tests for:
* mismatched row counts between X and X_embedded
* convert_dtype=False rejecting non-float32 input
* 1D input rejection (ensure_2d via check_array)
* unknown metric
@csadorf
csadorf force-pushed the issue-7998-apply-new-validation-to-metrics-trustworthiness branch from 1678d35 to 8c1b111 Compare May 7, 2026 17:09
@github-actions github-actions Bot added the Cython / Python Cython or Python issue label May 7, 2026

@jcrist jcrist left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@jcrist

jcrist commented May 7, 2026

Copy link
Copy Markdown
Member

/merge

@rapids-bot
rapids-bot Bot merged commit a70d355 into NVIDIA:main May 7, 2026
93 checks passed
@csadorf
csadorf deleted the issue-7998-apply-new-validation-to-metrics-trustworthiness branch May 7, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Cython / Python Cython or Python issue improvement Improvement / enhancement to an existing function non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants