Skip to content

Fix type reflection when input is an array-like - #8234

Merged
rapids-bot[bot] merged 8 commits into
NVIDIA:mainfrom
jcrist:infer-output-type
Jun 5, 2026
Merged

Fix type reflection when input is an array-like#8234
rapids-bot[bot] merged 8 commits into
NVIDIA:mainfrom
jcrist:infer-output-type

Conversation

@jcrist

@jcrist jcrist commented Jun 5, 2026

Copy link
Copy Markdown
Member

Our type reflection routines no longer match the implementation now that we support array-like (e.g. list) inputs to cuml methods. The routines in cuml.internals.validation treat these as if they were numpy arrays (with respect to how mem_type=None classifies them).

Due to this, inference on array-like inputs would fail, since determine_array_type wouldn't support array-like inputs, causing errors in our reflection machinery after the method had already successfully run.

This:

  • Adds a new infer_output_type function for inferring a corresponding output_type from a given input array-like. This new method is much simpler than the previous function that did this, and more closely matches how our new input validation routines treat input types. It's also in the same file as all the other reflection machinery, keeping the code closer together.
  • Adds tests for this. The old routine was untested, we now have good coverage of both the specific routine and also how it interacts with type reflection.
  • Applies infer_output_type everywhere determine_array_type used to be used. This fixed several bugs in test_sklearn_compatibility.
  • Deletes determine_array_type and related dead code in input_utils.py

jcrist added 7 commits June 5, 2026 09:00
This adds a new `infer_output_type` method for mapping an input to its
corresponding output type.

This new method is:

- Simpler than the old one in `input_utils.py`
- Maps better to how the new ingest `check_array` function treats inputs
- Properly handles array-like inputs, rather than returning `None`
@jcrist jcrist self-assigned this Jun 5, 2026
@jcrist
jcrist requested a review from a team as a code owner June 5, 2026 15:04
@jcrist
jcrist requested a review from betatim June 5, 2026 15:04
@jcrist jcrist added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Jun 5, 2026
@github-actions github-actions Bot added the Cython / Python Cython or Python issue label Jun 5, 2026
@coderabbitai

coderabbitai Bot commented Jun 5, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cd153ee0-3efa-474c-8aff-b74356c7435b

📥 Commits

Reviewing files that changed from the base of the PR and between fd1a135 and 4567115.

📒 Files selected for processing (2)
  • python/cuml/cuml/internals/outputs.py
  • python/cuml/tests/test_reflection.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • python/cuml/tests/test_reflection.py
  • python/cuml/cuml/internals/outputs.py

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes

    • Improved output-type inference and array conversion behavior for a wider range of array-like inputs, including sparse and GPU-backed arrays.
  • Tests

    • Expanded tests for type inference and reflection behavior across numpy/cupy/cudf/cuml and sparse inputs.
    • Adjusted compatibility test expectations to match improved array handling.
  • Chores

    • Removed an internal helper from the package exports and simplified input/type utility paths.

Walkthrough

This PR refactors cuML's input/output type inference by introducing a centralized infer_output_type function to replace scattered type-classification helpers. It removes deprecated functions (determine_array_type_full, input_to_host_array_with_sparse_support) and migrates call sites in outputs.py, base.py, and array_descriptor.py to the new inference path. Tests and sklearn compatibility xfails were updated.

Changes

Array type inference refactor

Layer / File(s) Summary
New output-type inference implementation
python/cuml/cuml/internals/outputs.py
Introduces infer_output_type(array, array_like="numpy") to classify inputs into numpy, cupy, cuml, cudf, pandas, numba, or None, using explicit type checks and heuristic detection via __array__, __array_interface__, and __len__ while excluding non-array collections and numpy scalars.
Remove deprecated type-inference helpers
python/cuml/cuml/internals/input_utils.py, python/cuml/cuml/common/__init__.py
Removes determine_array_type_full and input_to_host_array_with_sparse_support functions plus global_settings module variable from input_utils.py. Removes input_to_host_array_with_sparse_support re-export from cuml.common namespace.
Refactor outputs.py: coerce_arrays and reflect
python/cuml/cuml/internals/outputs.py
Updates coerce_arrays to use infer_output_type with inline sparse detection, converting inputs via CumlArray.from_input(order="K") or SparseCumlArray. Updates reflect's external output-type inference to call infer_output_type(array_arg).
Update array_descriptor.py: type inference and caching
python/cuml/cuml/common/array_descriptor.py
Imports infer_output_type for CumlArrayDescriptor.__set__ input-type computation. Updates cached internal array construction from input_to_cuml_array(...).array to CumlArray.from_input(..., order="K").
Update base.py: estimator output-type derivation
python/cuml/cuml/internals/base.py
Switches Base._set_output_type and Base._get_output_type to use infer_output_type(inp) instead of determine_array_type(inp). Updates imports accordingly.
Test coverage: infer_output_type and reflection behavior
python/cuml/tests/test_reflection.py
Adds comprehensive tests for infer_output_type across input types and interface detection. Introduces helper classes implementing array interfaces. Updates DummyEstimator to use check_inputs(..., reset=True) and @reflect(reset="type"). Tests verify array-likes are treated as numpy and remain unconverted when reflected functions return them.
Sklearn compatibility test cleanup
python/cuml/tests/test_sklearn_compatibility.py
Removes "data not an array" xfail checks from multiple estimators since improved type inference now handles array-like inputs. Updates remaining xfails to reflect specific observed failures (e.g., float32/float64 discrepancies).

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • rapidsai/cuml#7894: Modifies outputs.py's reflect flow and reset handling, directly preceding this PR's refactor of the same output-type logic path.
  • rapidsai/cuml#8042: Changes fit/validation and @reflect reset handling consistent with the new inference/check_inputs approach used here.
  • rapidsai/cuml#7753: Updates test_sklearn_compatibility.py xfail mappings; overlaps with this PR's xfail cleanup.

Suggested labels

bug, sklearn-api-compat

Suggested reviewers

  • betatim
  • divyegala
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 18.18% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Fix type reflection when input is an array-like' directly describes the main problem being addressed: fixing type reflection to handle array-like inputs. It is concise, specific, and captures the core change.
Description check ✅ Passed The description clearly explains the problem (type reflection not supporting array-like inputs), the solution (new infer_output_type function), and implementation details (replacing determine_array_type, adding tests, removing dead code).
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@csadorf csadorf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

Comment thread python/cuml/cuml/internals/outputs.py
@jcrist

jcrist commented Jun 5, 2026

Copy link
Copy Markdown
Member Author

/merge

@rapids-bot
rapids-bot Bot merged commit 9526712 into NVIDIA:main Jun 5, 2026
94 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Cython / Python Cython or Python issue improvement Improvement / enhancement to an existing function non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants