Build and test with CUDA 13.0.0 by jameslamb · Pull Request #366 · NVIDIA/cuopt

jameslamb · 2025-09-02T19:45:46Z

Description

Contributes to rapidsai/build-planning#208

uses CUDA 13.0.0 to build and test
moves some dependency pins:
- cuda-python: >=12.9.2 (CUDA 12), >=13.0.1 (CUDA 13)
- cupy: >=13.6.0
declares cuda-python runtime dependency for wheels (it was previously only declared for conda packages)

Contributes to rapidsai/build-planning#68

updates to CUDA 13 dependencies in fallback entries in dependencies.yaml matrices (i.e., the ones that get written to pyproject.toml in source control)

Notes for Reviewers

This switches GitHub Actions workflows to the cuda13.0 branch from here: rapidsai/shared-workflows#413

A future round of PRs will revert that back to branch-25.10, once all of RAPIDS supports CUDA 13.

Issue

Closes #294

Checklist

I am familiar with the Contributing Guidelines.
Testing
- New or existing tests cover these changes
- Added tests
- Created an issue to follow-up
- NA
Documentation
- The documentation is up to date with these changes
- Added new documentation
- NA

copy-pr-bot · 2025-09-02T19:45:50Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

jameslamb · 2025-09-02T20:53:47Z

cpp/src/mip/presolve/load_balanced_bounds_presolve_helpers.cuh

+                             &set_bounds_changed_node,  // "to" nodes
+#if CUDA_VER_13_0_UP
+                             nullptr,  // edge data
+#endif


Builds were failing like this:

│ │ $SRC_DIR/cpp/src/mip/presolve/load_balanced_bounds_presolve_helpers.cuh(474): error: argument of type "int" is incompatible with parameter of type "const cudaGraphEdgeData *"
│ │ cudaGraphAddDependencies(act_graph, &act_sub_warp_node, &set_bounds_changed_node, 1);

(conda-cpp-build)

Starting in CUDA 13, cudaGraphAddDependencies() picked up a new argument for optional edge data for the graphs.

ref:

https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__GRAPH.html#group__CUDART__GRAPH_1g575372c40c161728bfe500ebf676466c

Update cudaGraphAddDependencies for 13.0 cccl#5691

[STF] Fix CUDA graph API calls for CUDA 13 cccl#5636

This attempts to patch around that. This code was already not passing any data about edges in the graph, as far as I can tell, so hopefully this will be fine.

@rg20 @chris-maes for viz

After that change, there is a new build failure I don't understand:

│ │ $BUILD_PREFIX/bin/../lib/gcc/aarch64-conda-linux-gnu/14.3.0/../../../../aarch64-conda-linux-gnu/bin/ld: tmpxft_0000132e_00000000-6_execute_insertion.compute_120.cudafe1.cpp:(.text._ZN5cuopt7routing6detail24guided_ejection_search_tIifLNS0_9request_tE1EE40execute_best_insertion_ejection_solutionEPNS1_14request_info_tIiLS3_1EvEERi[_ZN5cuopt7routing6detail24guided_ejection_search_tIifLNS0_9request_tE1EE40execute_best_insertion_ejection_solutionEPNS1_14request_info_tIiLS3_1EvEERi]+0x52c): undefined reference to void cuopt::routing::detail::kernel_get_best_insertion_ejection_solution<64, int, float, (cuopt::routing::request_t)1>(cuopt::routing::detail::solution_t<int, float, (cuopt::routing::request_t)1>::view_t, cuopt::routing::detail::request_info_t<int, (cuopt::routing::request_t)1, void> const*, int*, int, int, cuopt::routing::detail::feasible_move_t, long)' │ │ $BUILD_PREFIX/bin/../lib/gcc/aarch64-conda-linux-gnu/14.3.0/../../../../aarch64-conda-linux-gnu/bin/ld: tmpxft_0000132e_00000000-6_execute_insertion.compute_120.cudafe1.cpp:(.text._ZN5cuopt7routing6detail24guided_ejection_search_tIifLNS0_9request_tE1EE40execute_best_insertion_ejection_solutionEPNS1_14request_info_tIiLS3_1EvEERi[_ZN5cuopt7routing6detail24guided_ejection_search_tIifLNS0_9request_tE1EE40execute_best_insertion_ejection_solutionEPNS1_14request_info_tIiLS3_1EvEERi]+0x680): undefined reference to void cuopt::routing::detail::kernel_get_best_insertion_ejection_solution<64, int, float, (cuopt::routing::request_t)1>(cuopt::routing::detail::solution_t<int, float, (cuopt::routing::request_t)1>::view_t, cuopt::routing::detail::request_info_t<int, (cuopt::routing::request_t)1, void> const*, int*, int, int, cuopt::routing::detail::feasible_move_t, long)'
│ │ $BUILD_PREFIX/bin/../lib/gcc/aarch64-conda-linux-gnu/14.3.0/../../../../aarch64-conda-linux-gnu/bin/ld: libcuopt.so: hidden symbol `_ZN5cuopt18linear_programming6detail33update_changed_constraints_kernelIidEEvNS1_4fj_tIT_T0_E14climber_data_t6view_tE' isn't defined

(conda-cpp-build link)

@rgsl888prabhu @rg20 @chris-maes are any of you available to help with me with this? You're welcome to push commits directly to my branch here.

I will try to get to this tmrw. Thank you @jameslamb for the help on the CUDA 13

I am working on this, setting up build environment now.

@jameslamb I have pushed changes which fixed the cpp build, I think wheels needs to be worked on.

Excellent, thanks!

Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

(wheel-build-cuopt-mps-parser link)

I think I can fix this.

Signed-off-by: Ramakrishna Prabhu <ramakrishnap@nvidia.com>

jameslamb · 2025-09-05T04:13:02Z

.github/workflows/build.yaml

      append-cuda-suffix: false
+      pure-wheel: true
+      # only need 1 build (noarch package): this selects amd64, oldest-supported Python, latest-supported CUDA
+      matrix_filter: '[map(select(.ARCH == "amd64")) | min_by((.PY_VER | split(".") | map(tonumber)), (.CUDA_VER | split(".") | map(-tonumber)))]'


(moved to a corrected version of this filter)

Changes like this fix the errors about conflicting artifact names. For pure-Python packages, we only need to build 1 time... not once per combination of Python version, CUDA versions, operating system, and CPU architecture.

This change should save cuopt some CI time by avoiding lots of unnecessary wheel builds 😁

…t-sh-client is)

jameslamb · 2025-09-05T16:02:27Z

docs/cuopt/source/cuopt-c/quick-start.rst

-      nvidia-cuda-runtime-cu12==12.9.* \
-      libcuopt-cu12==25.10.*
+      'nvidia-cuda-runtime-cu12==12.9.*' \
+      'libcuopt-cu12==25.10.*'


Touching these because they came up in merge conflicts with #367

You have to single-quote specifiers with special shell characters like <, >, or * .... otherwise they may be treated specially by some shells and do surprising things.

rgsl888prabhu

@jameslamb Awesome work and thank you for handling this, have a minor suggestion for the doc, but rest looks good with respect to infra.

docs/cuopt/source/cuopt-c/quick-start.rst

hlinsen

cpp changes look good. Thanks @jameslamb!

jameslamb · 2025-09-05T17:47:48Z

Thank you both!!! I'll merge this once CI passes.

jameslamb · 2025-09-05T18:55:38Z

/merge

Contributes to rapidsai/build-planning#208 Updates the `:latest` and `:25.10-latest` tags to CUDA 13.0.0. ## Notes for Reviewers ### is this safe to merge? Once these are in, I think yes: * [x] NVIDIA/cuopt#366 * [x] rapidsai/cugraph-gnn#286 At that point, the only thing it should affect are docs builds across repos that are already supporting CUDA 13 in all their other conda-based tests. Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Kyle Edwards (https://github.com/KyleFromNVIDIA) URL: #303

cwilkinson76

LGTM

Contributes to rapidsai/build-planning#208 * updates all GitHub Actions branch references back to `branch-25.10`, now that rapidsai/shared-workflows#413 is merged * fixes docs mistakes (#366 (comment)) * fixes nightly builds Nightly wheel builds of `cuopt-mps-parser` are failing like this: > Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run ([wheel-build-cuopt-mps-parser link](https://github.com/NVIDIA/cuopt/actions/runs/17501959410/job/49716771454)) Because I forgot to bring over all of the artifact-naming changes made in `pr.yaml` to the corresponding entries on `build.yaml`, sorry 😬 Authors: - James Lamb (https://github.com/jameslamb) - Ramakrishnap (https://github.com/rgsl888prabhu) Approvers: - Ramakrishnap (https://github.com/rgsl888prabhu) URL: #377

Contributes to rapidsai/build-planning#208 * uses CUDA 13.0.0 to build and test * moves some dependency pins: - `cuda-python`: `>=12.9.2` (CUDA 12), `>=13.0.1` (CUDA 13) - `cupy`: `>=13.6.0` * declares `cuda-python` runtime dependency for wheels ([it was previously only declared for conda packages](https://github.com/NVIDIA/cuopt/blob/c62320447414c47f25ea67bb2570e05c7d0d29ac/conda/recipes/cuopt/recipe.yaml#L70)) Contributes to rapidsai/build-planning#68 * updates to CUDA 13 dependencies in fallback entries in `dependencies.yaml` matrices (i.e., the ones that get written to `pyproject.toml` in source control) ## Notes for Reviewers This switches GitHub Actions workflows to the `cuda13.0` branch from here: rapidsai/shared-workflows#413 A future round of PRs will revert that back to `branch-25.10`, once all of RAPIDS supports CUDA 13. ## Issue Closes #294 Authors: - James Lamb (https://github.com/jameslamb) - Ramakrishnap (https://github.com/rgsl888prabhu) Approvers: - Ramakrishnap (https://github.com/rgsl888prabhu) - Hugo Linsenmaier (https://github.com/hlinsen) URL: #366

Contributes to rapidsai/build-planning#208 * updates all GitHub Actions branch references back to `branch-25.10`, now that rapidsai/shared-workflows#413 is merged * fixes docs mistakes (#366 (comment)) * fixes nightly builds Nightly wheel builds of `cuopt-mps-parser` are failing like this: > Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run ([wheel-build-cuopt-mps-parser link](https://github.com/NVIDIA/cuopt/actions/runs/17501959410/job/49716771454)) Because I forgot to bring over all of the artifact-naming changes made in `pr.yaml` to the corresponding entries on `build.yaml`, sorry 😬 Authors: - James Lamb (https://github.com/jameslamb) - Ramakrishnap (https://github.com/rgsl888prabhu) Approvers: - Ramakrishnap (https://github.com/rgsl888prabhu) URL: #377

jameslamb added 2 commits September 2, 2025 14:36

Build and test with CUDA 13.0.0

3c6ef83

declare cuda-python dependency for wheels

a515be4

jameslamb added non-breaking Introduces a non-breaking change improvement Improves an existing functionality labels Sep 2, 2025

jameslamb mentioned this pull request Sep 2, 2025

Add CUDA 13.0 Support across all of RAPIDS rapidsai/build-planning#208

Closed

patch cudaGraphAddDependencies() uses

7b904b7

jameslamb commented Sep 2, 2025

View reviewed changes

set(CMAKE_CUDA_FLAGS

eac1ca6

Signed-off-by: Ramakrishna Prabhu <ramakrishnap@nvidia.com>

anandhkb added this to the 25.10 milestone Sep 4, 2025

jameslamb added 4 commits September 3, 2025 22:50

fix merge conflicts, update to latest branch-25.10

0115174

remove unnecessary builds

4517f5f

restore guard

92b8192

fix matrix_filter syntax

7275d7c

NVIDIA deleted a comment from rgsl888prabhu Sep 4, 2025

jameslamb changed the title ~~WIP: Build and test with CUDA 13.0.0~~ Build and test with CUDA 13.0.0 Sep 4, 2025

fix matrix filter again

589981e

jameslamb marked this pull request as ready for review September 4, 2025 15:03

jameslamb requested review from a team as code owners September 4, 2025 15:03

jameslamb requested review from Iroy30, KyleFromNVIDIA, chris-maes, rg20 and rgsl888prabhu and removed request for Iroy30, KyleFromNVIDIA, chris-maes and rg20 September 4, 2025 15:03

jameslamb added 2 commits September 4, 2025 22:26

ah, ok, this is how you do pure-python wheel builds

089b10e

typo

65175c2

jameslamb commented Sep 5, 2025

View reviewed changes

jameslamb changed the title ~~WIP: Build and test with CUDA 13.0.0~~ Build and test with CUDA 13.0.0 Sep 5, 2025

jameslamb added 4 commits September 4, 2025 23:31

ah, cuopt-mps-parser has Cython code so is not a pure wheel (but cuop…

c542de4

…t-sh-client is)

more matrix filter

43e4322

tested this matrix_filter, I feel good about it

00112bc

bump libcuopt wheel size limit to 800M

28afb4f

jameslamb mentioned this pull request Sep 5, 2025

update 'latest' to CUDA 13 rapidsai/ci-imgs#303

Merged

2 tasks

jameslamb added 2 commits September 5, 2025 10:30

another size increase

1ddd504

Merge branch 'branch-25.10' of github.com:NVIDIA/cuopt into cuda-13.0.0

2f44d9e

jameslamb commented Sep 5, 2025

View reviewed changes

jameslamb mentioned this pull request Sep 5, 2025

add CUDA 13.0 rapidsai/shared-workflows#413

Merged

2 tasks

jameslamb removed the do not merge Do not merge if this flag is set label Sep 5, 2025

one more missed PURE_WHEEL thing for cuopt-mps-parser

6c38e78

rgsl888prabhu approved these changes Sep 5, 2025

View reviewed changes

docs/cuopt/source/cuopt-c/quick-start.rst Outdated Show resolved Hide resolved

Update docs/cuopt/source/cuopt-c/quick-start.rst

c458423

rgsl888prabhu requested review from hlinsen and rg20 September 5, 2025 17:37

hlinsen approved these changes Sep 5, 2025

View reviewed changes

rapids-bot bot merged commit af47d49 into NVIDIA:branch-25.10 Sep 5, 2025
102 checks passed

jameslamb deleted the cuda-13.0.0 branch September 5, 2025 18:55

cwilkinson76 approved these changes Sep 5, 2025

View reviewed changes

jameslamb mentioned this pull request Sep 5, 2025

CUDA 13 support: follow-ups #377

Merged

Comments

Conversation

jameslamb commented Sep 2, 2025

Description

Notes for Reviewers

Issue

Checklist

Uh oh!

copy-pr-bot bot commented Sep 2, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rgsl888prabhu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hlinsen left a comment

Choose a reason for hiding this comment

Uh oh!

jameslamb commented Sep 5, 2025

Uh oh!

jameslamb commented Sep 5, 2025

Uh oh!

Uh oh!

cwilkinson76 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants