Large (int64 indices) sparse input support

[jcrist]

This:

• Updates some code in cuml.internals.validation to provide a better error message if cupy < 14.1.0 when users pass in large sparse matrices on host, noting the cupy version that's required to support this.
• Adds some tests for our validation routines for large device-side sparse matrices. This is possible to do in low memory.
• Sets accept_large_sparse=True for Ridge, LinearRegression, and cuml.random_projection. I've tested that these work locally, but running these requires more memory than we can feasibly rely on in CI. The approach used in sklearn (constructing a tiny sparse matrix, but forcing int64 indices) isn't feasible here, since cupy rightfully tries to optimistically coerce indices to int32 values, which would defeat the purpose of testing. Since int32 indices run much faster, I think this validation check is valid and not something worth working around. FWIW, these estimators are mostly plumbing the inputs -> routines in cupy/cublas/cusparse/cusolver, so I don't think missing tests in CI here should be a release blocker.

Fixes #8150.
Fixes #8151.
Fixes #8152.
Fixes #8157.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b973375d-1013-4e5e-be73-83c7290fc6e6

📥 Commits

Reviewing files that changed from the base of the PR and between ca8db91 and 5d259f0.

📒 Files selected for processing (1)

• python/cuml/cuml/internals/validation.py

🚧 Files skipped from review as they are similar to previous changes (1)

• python/cuml/cuml/internals/validation.py

---

📝 Walkthrough

Summary by CodeRabbit

• New Features
  • Linear regression, ridge regression, and random projection now accept larger sparse matrices with int64 indices when the installed CuPy supports it (>= 14.1.0).
• Behavior Changes
  • Validation now explicitly rejects unsupported large sparse inputs on older CuPy versions with a clear error message.
• Data Handling
  • DataFrame float16 columns are consistently upcast to float32 during input validation.
• Tests
  • Added tests covering large-sparse input acceptance and rejection scenarios.

Walkthrough

Adds an int64-sparse detector and CuPy >=14.1.0 gate to validation, opts estimators into large-sparse acceptance, adds version-gated tests, and simplifies float16 upcasting in cudf handling.

Changes

Large Sparse Matrix Support

Layer / File(s)	Summary
Sparse matrix validation infrastructure and CuPy version gating python/cuml/cuml/internals/validation.py	Adds _CUPY_SUPPORTS_LARGE_SPARSE gate, _requires_int64_sparse detector, refactors _ensure_int32_sparse to use it, updates check_array to reject device-side int64-index sparse inputs on older CuPy, and tweaks a host-conversion comment.
Large sparse support integration across estimators python/cuml/cuml/linear_model/linear_regression.pyx, python/cuml/cuml/linear_model/ridge.pyx, python/cuml/cuml/random_projection/random_projection.py	LinearRegression.fit, Ridge.fit, _BaseRandomProjection.fit, and _BaseRandomProjection.transform now pass accept_large_sparse=True to check_inputs to allow large sparse inputs during validation.
Test coverage for large sparse matrix support python/cuml/tests/test_validation.py	Adds Version import and CUPY_SUPPORTS_LARGE_SPARSE flag; two new check_array tests are gated by CuPy version to validate acceptance on supported versions and the upgrade-required error on older versions.
Float16 handling simplification python/cuml/cuml/internals/validation.py	Simplifies check_cudf float16-column upcasting by removing pandas-version branching and always using astype float16→float32.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

• Add support for large sparse matrices to check_array #8150: Adds large sparse matrix test and explicit error path for older CuPy — aligns with the validation gating and tests added here.
• Support large sparse matrices in Ridge #8151: Requests large sparse support in Ridge — this PR passes accept_large_sparse=True in Ridge.fit.
• Support large sparse matrices in LinearRegression #8152: Requests large sparse support in LinearRegression — this PR passes accept_large_sparse=True in LinearRegression.fit.
• Support large sparse matrices in cuml.random_projection #8157: Requests large sparse support in cuml.random_projection — this PR updates random_projection fit/transform validation.

Possibly related PRs

• rapidsai/cuml#8009: Modifies random_projection.py fit/transform input handling via check_inputs, closely related to the random-projection changes here.
• Use new input validation infrastructure for cuml.decomposition. #8006: Updates shared validation utilities and uses accept_large_sparse in related estimator adaptations.
• Cleanup and apply new validation to cuml._thirdparty and cuml.preprocessing #8052: Changes shared validation utilities; overlaps with check_array/check_inputs edits.

Suggested labels

algo: linear-model

Suggested reviewers

• divyegala
• csadorf

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 72.73% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: adding support for large (int64 indices) sparse input across the codebase.
Description check	✅ Passed	The description relates to the changeset by explaining updates to validation, adding tests, and enabling large sparse support across multiple estimators.
Linked Issues check	✅ Passed	The PR addresses all four linked issues: #8150 (validation & tests for large sparse), #8151 (Ridge), #8152 (LinearRegression), and #8157 (random_projection) by adding accept_large_sparse=True and improved error handling.
Out of Scope Changes check	✅ Passed	All changes are directly aligned with linked issue objectives: validation improvements, test additions, and enabling large sparse support in the specified estimators.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/cuml/tests/test_validation.py`:
- Around line 1133-1135: The pytest.raises call currently uses match="Sparse
matrices with int64 indices require cupy >= 14.1.0" which is treated as a regex;
escape the literal dots so the version string is matched literally (e.g.,
replace "." with "\." in the version portion) in the pytest.raises match
argument within the test that contains this raise check to ensure the assertion
matches the exact message.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b4e7cb6b-7aeb-414f-bb25-2b9464a559e3

📥 Commits

Reviewing files that changed from the base of the PR and between 831b8a0 and ca8db91.

📒 Files selected for processing (5)

• python/cuml/cuml/internals/validation.py
• python/cuml/cuml/linear_model/linear_regression.pyx
• python/cuml/cuml/linear_model/ridge.pyx
• python/cuml/cuml/random_projection/random_projection.py
• python/cuml/tests/test_validation.py

[csadorf]

Great! Just two minor things.

[jcrist]

/merge