Add LSMR solver to Ridge - #7922
Merged
Merged
Conversation
jcrist
added
feature request
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughAdds sparse-input support and an LSMR solver to Ridge, introduces a center_and_scale utility, exposes tol/max_iter and n_iter_, updates sklearn-compat GPU validation for sparse/solver combos, adjusts prediction indexing for converted inputs, and updates tests and docs accordingly. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@python/cuml/cuml/linear_model/ridge.pyx`:
- Around line 558-562: The ValueError message in the conditional that checks
X_is_sparse and solver != "lsmr" in ridge.pyx is missing the f-string prefix so
"{solver!r}" won't be interpolated; update the string literal in that raise
inside the if block to use an f-string (prefix with f) so solver is formatted
into the message (refer to the conditional using X_is_sparse and the variable
solver to find the exact spot).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 7278859d-49d5-48ac-af07-c64ecad6fd13
📒 Files selected for processing (5)
docs/source/cuml-accel/limitations.rstpython/cuml/cuml/accel/_overrides/sklearn/linear_model.pypython/cuml/cuml/linear_model/base.pypython/cuml/cuml/linear_model/ridge.pyxpython/cuml/tests/test_ridge.py
💤 Files with no reviewable changes (1)
- docs/source/cuml-accel/limitations.rst
Member
Author
|
A quick benchmark on dense inputs: from time import perf_counter
import cuml
from cuml.datasets import make_regression
for n_samples in [5_000, 10_000, 20_000]:
for n_features in [1_000, 5_000, 10_000]:
X, y = make_regression(
n_samples, n_features, random_state=42, dtype="float"
)
print(f"shape: {X.shape}")
for solver in ["eig", "svd", "lsmr"]:
start = perf_counter()
cuml.Ridge(solver=solver, tol=1e-6).fit(X, y)
duration = perf_counter() - start
print(f"- {solver}: {duration:.3f} s")Output LSMR is typically the fastest across dimensionality. For small I'm not sure if we'd want to make |
There was a problem hiding this comment.
🧹 Nitpick comments (2)
python/cuml/tests/test_ridge.py (2)
140-141: Consider clarifyingn_targets=0semantics.Using
n_targets=0to represent the 1D-y (single-target) case is functional but could be confusing to readers. A brief comment explaining this convention would improve clarity.`@pytest.mark.parametrize`("n_targets", [0, 1, 3]) def test_ridge_sparse(dtype, fit_intercept, weighted, n_targets): + # n_targets=0 tests 1D y (single target), n_targets=1 tests 2D y with 1 column n_samples, n_features = 3000, 500🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@python/cuml/tests/test_ridge.py` around lines 140 - 141, The test parameterization uses n_targets with a special convention where n_targets=0 denotes the 1D-y (single-target) case; add a brief clarifying comment above the pytest.mark.parametrize or at the start of test_ridge_sparse explaining that n_targets==0 is used to represent a single-target (1D) response to avoid confusion, and reference the parameter name n_targets and test function test_ridge_sparse so readers understand the mapping and why 0 is used.
159-166: Consider matchingtolbetween cuML and sklearn for fair comparison.
cuml.Ridgeis initialized withtol=1e-6, butsklearn.linear_model.Ridgeuses its defaulttol=1e-4. While theatol=1e-2assertion tolerance is permissive enough to pass, matching tolerances would make the comparison more meaningful.cu_model = cuml.Ridge(tol=1e-6, fit_intercept=fit_intercept) ... - sk_model = sklearn.linear_model.Ridge( - solver="lsqr", fit_intercept=fit_intercept - ) + sk_model = sklearn.linear_model.Ridge( + solver="lsqr", fit_intercept=fit_intercept, tol=1e-6 + )🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@python/cuml/tests/test_ridge.py` around lines 159 - 166, The test uses different tolerances between cuml.Ridge (cu_model with tol=1e-6) and sklearn.linear_model.Ridge (sk_model using default tol=1e-4); update the sklearn model initialization to pass tol=1e-6 so sk_model and cu_model use the same convergence tolerance for a fair comparison (i.e., set tol in the sklearn.linear_model.Ridge constructor to match the tol used when creating cu_model).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@python/cuml/tests/test_ridge.py`:
- Around line 140-141: The test parameterization uses n_targets with a special
convention where n_targets=0 denotes the 1D-y (single-target) case; add a brief
clarifying comment above the pytest.mark.parametrize or at the start of
test_ridge_sparse explaining that n_targets==0 is used to represent a
single-target (1D) response to avoid confusion, and reference the parameter name
n_targets and test function test_ridge_sparse so readers understand the mapping
and why 0 is used.
- Around line 159-166: The test uses different tolerances between cuml.Ridge
(cu_model with tol=1e-6) and sklearn.linear_model.Ridge (sk_model using default
tol=1e-4); update the sklearn model initialization to pass tol=1e-6 so sk_model
and cu_model use the same convergence tolerance for a fair comparison (i.e., set
tol in the sklearn.linear_model.Ridge constructor to match the tol used when
creating cu_model).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 26d709a0-3115-4013-8134-8a756ec12f07
📒 Files selected for processing (2)
python/cuml/cuml/linear_model/ridge.pyxpython/cuml/tests/test_ridge.py
viclafargue
approved these changes
Mar 23, 2026
This function will be reused later on for other linear models, so taking care of it first.
This adds a new LSMR solver to `Ridge`. The solver supports sparse and dense inputs, multi-target regressions, and is typically much faster than either existing solvers (eig or svd). For now we kept `'auto'` defaulting the same as before, except in the case of sparse inputs we now use `lsmr` instead of erroring.
Member
Author
|
/merge |
rapids-bot Bot
pushed a commit
that referenced
this pull request
Apr 1, 2026
This is a followup to #7922. It adds an LSMR solver to `LinearRegression`. The primary motivation here is adding sparse input support, but the LSMR solver is also much faster than our existing solvers for most inputs. For now I kept the default behavior the same (except we use LSMR for sparse inputs) - in the future we _might_ consider preferring LSMR over SVD. To accomplish this, I split the cupy-based solvers out of `cuml/linear_models/ridge.pyx` into `cuml/linear_models/base.py` as a standalone `fit_least_squares` function. This feels a _bit_ weird to have it there, but it's nice to have pure-python functions in a `.py` file since we get much better linting/formatting there than we do in cython files. I'm happy with this location for now. Since `LinearRegression` is effectively a special-case of `Ridge` with `alpha=0.0`, sharing this functionality across the models makes sense. Fixes #3105. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Victor Lafargue (https://github.com/viclafargue) URL: #7927
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a new
solver="lsmr"toRidge, based oncupyx.scipy.sparse.linalg.lsmr. This solver supports both sparse and dense inputs, multi-target regressions, and is typically much faster than either existing solver.For now we keep the
solver="auto"default the same as before, with the exception that for sparse inputs we now default to'lsmr'instead of erroring.For cuml-accel, we map
solver="lsqr"tosolver="lsmr", since they're closely related.Fixes #7911.