Forward-merge release/26.06 into main - #8116
Merged
Merged
Conversation
This is required to import `libcuml`, but wasn't explicitly listed as a dependency. The parent main user-facing package (`cuml`) does have a `rmm` dependency, which in turn depends on `librmm`, so everything would be installed properly normally anyway. Just adding an explicit link here for tidyness. Part of #7845. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Kyle Edwards (https://github.com/KyleFromNVIDIA) URL: #8110
Contributor
Author
|
FAILURE - Unable to forward-merge due to an error, manual merge is necessary. Do not use the IMPORTANT: When merging this PR, do not use the auto-merger (i.e. the |
Member
|
Fixing the merge conflict in #8119. |
Our pipeline data transfer optimization didn't work if any of the steps were other compositional metaestimators that wrapped accelerated estimators (since these could then accidentally use the accelerated versions, resulting in a mix of `cupy` and `numpy` results). This PR patches the other two compositional estimators (`FeatureUnion` and `ColumnTransformer`) so they always run within a `numpy` output-type context. Fixes #8112. Fixes a few sklearn examples as well (yay!) Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Simon Adorf (https://github.com/csadorf) URL: #8115
This configures cupy to use the per-thread default stream (PTDS) for _most_ operations. This avoids usage of the default legacy stream in more of the codebase, allowing for improved parallelism when running across multiple threads. **This is a breaking change.** Previously any cupy operations in `cuml` ran in cupy's default stream (the legacy stream). We didn't synchronize the stream before returning, but that didn't matter due to the synchronization behavior of the legacy stream. With this PR we've moved to running (most) cupy operations in the PTDS. Depending on the operation, we may not synchronize the PTDS before returning. **Most users shouldn't notice a difference and should have no issues.** Users not using threads, custom streams, or only working with host memory (e.g. numpy in/numpy out) should see no difference. Likewise any users that only use cupy's default stream (the legacy stream) in their code should see no issues. Users doing tricky things with custom streams or threads may run into issues and require a manual sync of the PTDS (can be done with `cupy.cuda.Stream.ptds.synchronize()`. For example, the following workflow _may_ run into issues: - Run a cuml operation based on cupy in thread A, returning a cupy array - Consume that output in thread B as a cupy array using a stream other than the legacy stream (e.g. a different PTDS or a custom stream) For safety, you probably want to add a call to `cupy.cuda.Stream.ptds.synchronize()` in thread A before returning to ensure the output array is fully populated before consuming it in thread B. Fixes #7909. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Simon Adorf (https://github.com/csadorf) - Dante Gama Dessavre (https://github.com/dantegd) URL: #8086
This adds to the cuml.accel documentation and documents the "third party application" use-case. Authors: - Tim Head (https://github.com/betatim) Approvers: - Jim Crist-Harif (https://github.com/jcrist) URL: #8094
This: - Bumps our minimum supported `numpy` version to 1.26, to match that of `cudf`. Since `cudf` is a required dependency, we were effectively pinned at that already. - Adds `numpy` to our oldest deps test runs. This also effectively adds `cupy==13.6`, since `cupy==14` requires `numpy>=2.0`. Explicitly specifying `cupy==13.6` in an oldest-deps run is tricky since the pypi packages require cuda suffixes as well. I'm skipping that for now. - Fixes a few incompatibilities with numpy 1.x Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Gil Forsyth (https://github.com/gforsyth) - Simon Adorf (https://github.com/csadorf) URL: #8118
Closes #8121 Proposes slightly reducing the threshold in the R-squared check for `MBSGDRegressor` tests. Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Simon Adorf (https://github.com/csadorf) URL: #8122
This adds a cuml.accel example that illustrates the point that using cuml.accel makes it easier to do things because you aren't interrupted by things taking forever and forever. It uses a simple pipeline that can be fully GPU accelerated and then searches a few hyper-parameter combinations for that pipeline. Authors: - Tim Head (https://github.com/betatim) Approvers: - Jim Crist-Harif (https://github.com/jcrist) URL: #8095
This has started to fail. xfailing for now until the issue can be investigated. Stopgap for #8125. Fixes #8129. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Simon Adorf (https://github.com/csadorf) URL: #8127
The dense version is already marked as flaky, but I've now seen the sparse version fail twice. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Simon Adorf (https://github.com/csadorf) URL: #8130
This addresses failures in `python/cuml/tests` and `python/cuml/cuml_accel_tests` when run with the most recent sklearn 1.9 pre-release. With one small exception, this just required a few tweaks to some tests. I did not address any failures when running the upstream sklearn test suite with `cuml.accel`, as that would require much deeper changes to setup xfails. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Simon Adorf (https://github.com/csadorf) URL: #8126
- Updates `LabelBinarizer` to follow standard cuml and sklearn conventions (simple `__init__`, no mutation, type reflection, ...) - Applies new validation - Adds `sparse_input_` and `y_type_` attributes - Improves validation and error messages - Improves resilience and sklearn compatibility - Improves test coverage - Adds support for sklearn interop - Adds support for cuml.accel - Improves docstrings This required one change to `cuml.internals.validation` around handling of unsupported dtypes for `cupyx.scipy.sparse`. This is split out into a separate commit with a new test case. Part of #7317. Fixes #8087. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Simon Adorf (https://github.com/csadorf) URL: #8101
Closes #7982 Mirrors sklearn PR #32050 on cuml.SVC and cuml.LinearSVC. Sentinel + _effective_X property pattern, same as PR #7958. FutureWarning fires from fit when the user passes an explicit value. Also fixes a latent bug in the accel proxy where _gpu_fit was reading self.probability truthily. Default SVC() on small data would have routed through the probability code path since the sentinel string is truthy. Authors: - Steve Collins (https://github.com/switch527) - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Jim Crist-Harif (https://github.com/jcrist) URL: #8089
This failure started after the recent CCCL upgrade and appears only on rtxpro6000 test runs. We relaxed the tolerance once already, but have still seen a rare periodic failure. From looking at recent failures, an atol of 5e-3 would be sufficient, but bumping to 1e-2 to be sure. We're only checking plumbing here, so some slop in tolerance is fine. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Simon Adorf (https://github.com/csadorf) - Victor Lafargue (https://github.com/viclafargue) URL: #8136
Adds `sklearn.decomposition.IncrementalPCA` to `cuml.accel`, including `fit_transform` and `partial_fit` dispatch through the estimator proxy. This also tightens sklearn parity for cuML `IncrementalPCA` fitted attributes, CPU/GPU interop, `var_` handling, first-batch validation behavior, and `set_output` support. Closes #7779 Authors: - John Zedlewski (https://github.com/JohnZed) - Simon Adorf (https://github.com/csadorf) Approvers: - Simon Adorf (https://github.com/csadorf) URL: #7785
Adds a new doc with some advanced topics. Includes: - CUDA streams and synchronization - CUDA device selection - A very small section on configuring RMM. I wanted to mention something, but not provide a full guide. Once the upstream RMM one lands we can link there. xref #8128 Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Gil Forsyth (https://github.com/gforsyth) - Simon Adorf (https://github.com/csadorf) URL: #8134
Adopts `nvForest` for cuML random forest inference, removes the in-tree FIL implementation, and keeps a deprecated `cuml.fil` compatibility layer that directs users to `nvforest.ForestInference`. This also wires `nvForest` into the C++ and Python build paths, updates packaging dependencies, and preserves the existing `cuml.fil.ForestInference` entry point with deprecation warnings for users migrating from FIL. Authors: - Philip Hyunsu Cho (https://github.com/hcho3) - Simon Adorf (https://github.com/csadorf) Approvers: - Simon Adorf (https://github.com/csadorf) - Jim Crist-Harif (https://github.com/jcrist) - Bradley Dice (https://github.com/bdice) URL: #8048
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Forward-merge triggered by push to release/26.06 that creates a PR to keep main up-to-date. If this PR is unable to be immediately merged due to conflicts, it will remain open for the team to manually merge. See forward-merger docs for more info.