remove RAPIDS dependencies from cu12, cu13 extras

[jameslamb]

#1536 introduced [cu12] and [cu13] extras for dask-cuda wheels, so that you could do something like

pip install 'dask-cuda[cu13]'

To ensure that numba-cuda[cu13] (and appropriately-pinned dependencies of it) are installed. In that PR, I'd also added RAPIDS CUDA-version-suffixed optional runtime dependencies to those extras, thinking:

• this is an extra... it's already optional
• one (and in my view, the main) reason to keep those out of the runtime dependencies before was to avoid needing a -cu{12,13} suffix here, but with the introduction of these extras that constraint was removed

However, that caused some problems for RAPIDS devcontainers. For example, cudf repo-specific devcontainers are now installing cudf-cu{12,13} from remote indices instead of building it from source (rapidsai/devcontainers#568 was an earlier attempt to fix that).

To avoid these circular-dependency problems, this proposes removing RAPIDS libraries from the [cu12] / [cu13] extras. That restores dask-cuda to the state it was in before, where using optional dependencies like cudf will require installing those separately.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[jameslamb]

/ok to test

[pentschev]

We still want numba-cuda to be installed for testing only, this ensures

dask-cuda/dask_cuda/tests/test_gds.py

Lines 30 to 32 in 224a407

	elif cuda_lib == "numba.cuda":
	data_create = lambda: lib.to_device(range(10))
	data_compare = lambda x, y: all(x.copy_to_host() == y.copy_to_host())

still runs.

[jameslamb]

numba-cuda to be installed for testing only

#1531 added numba-cuda as a runtime dependency, don't the reasons for that PR still hold, and therefore shouldn't numba-cuda continue to be a runtime dependency?

[pentschev]

Oh yes, that's right, the conda environment files got me confused, why is numba-cuda removed from there, shouldn't it be in there too to satisfy it?

[jameslamb]

ahhhh yes YOU are right!

When I saw those removals I thought "ok, makes sense, runtime dependencies get installed via the package metadata for dask-cuda, not pre-installed in the environment" but I don't think that's what we usually do in RAPIDS for those conda env files checked into the repo.

All the other runtime dependencies (like pandas) are in there.

I just pushed bfe3317 restoring it as a runtime dependency. Now numba-cuda will be in those conda env files, and it'll also show up in the wheel's strong runtime dependencies.

I think this should be ok, and is actually desirable.

# depends on: 'numba-cuda>=0.19.1,<0.20.0a0'
pip install dask-cuda

# depends on: 'numba-cuda>=0.19.1,<0.20.0a0', 'numba-cuda[cu12]>=0.19.1,<0.20.0a0'
pip install 'dask-cuda[cu12]'

Having one dependency with constraints and one without should be fine. Tested like this:

docker run \
  --rm \
  -it rapidsai/citestwheel:25.10-cuda12.9.1-ubuntu24.04-py3.13 \
  pip install 'numba-cuda>=0.19.1,<0.20.0a0' 'numba-cuda[cu12]>=0.19.1,<0.20.0a0'

And saw it succeed and respect numba-cuda's constraints, like this:

...
Collecting cuda-bindings<13.0.0,>=12.9.1 (from numba-cuda[cu12]<0.20.0a0,>=0.19.1)
  Downloading cuda_bindings-12.9.2-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (2.6 kB)
Collecting cuda-core==0.3.* (from numba-cuda[cu12]<0.20.0a0,>=0.19.1)
...

[pentschev]

No, but wait, we did make a decision to NOT depend on numba-cuda anymore in favor of cuda-bindings, no? I think I'm confused to why we need it to be a runtime in dask-cuda, presumably because of cudf-cu[12,13]? This is what I previously proposed, and I thought that's what we ultimately had done/intended to do in #1536 . Am I mixing things up again?

[jameslamb]

From that link I shared, it looks to me like cuda-core depends (optionally, via a [cu12,13] extra) on cuda-bindings, not the other way around.

https://github.com/NVIDIA/cuda-python/blob/fa07cf58ef5c70bb6a42e9aa859487e88f5699a2/cuda_core/pyproject.toml#L50-L51

Depending on cuda-bindings alone won't bring in cuda-core.

docker run \
  --rm \
  --gpus all \
  -it rapidsai/citestwheel:25.10-cuda12.9.1-ubuntu24.04-py3.13 \
  bash

# install 'cuda-bindings' (no version)
pip install cuda-bindings

Installing cuda-bindings without a version pulls in CUDA 13 versions of cuda-bindings and cuda-pathfinder:

Looking in indexes: https://pypi.org/simple, https://pypi.anaconda.org/rapidsai-wheels-nightly/simple, https://pypi.nvidia.com
Collecting cuda-bindings
  Downloading cuda_bindings-13.0.1-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (2.7 kB)
Collecting cuda-pathfinder~=1.1 (from cuda-bindings)
  Downloading cuda_pathfinder-1.2.1-py3-none-any.whl.metadata (3.1 kB)
Downloading cuda_bindings-13.0.1-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (12.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.3/12.3 MB 9.7 MB/s  0:00:01
Downloading cuda_pathfinder-1.2.1-py3-none-any.whl (22 kB)
Installing collected packages: cuda-pathfinder, cuda-bindings
Successfully installed cuda-bindings-13.0.1 cuda-pathfinder-1.2.1

And the cuda.core namespace is not available.

python -c "import cuda.core"
# ModuleNotFoundError: No module named 'cuda.core'

So I think we want the following for dask-cuda's runtime requirements

• dask-cuda -> cuda-core>=12.0,<14
• dask-cuda[cu12] -> cuda-core[cu12]==12.*
• dask-cuda[cu13] -> cuda-core[cu13]==13.*

[pentschev]

Yeah, that's what I wanted to say, cuda-bindings would allow a proper existing environment to work. It's looking more and more like there's no way around it, let's just pin cuda-core as you proposed, it seems that is the only correct pin, unfortunately. Could you do that?

[jameslamb]

Ok yep, I can do that!

[jameslamb]

Ah one clarification... cuda-core's versioning does not follow the CTK, sorry for that mistake.

So it'd be:

• dask-cuda -> cuda-core==0.3.*
• dask-cuda[cu12] -> cuda-core[cu12]==0.3.*
• dask-cuda[cu13] -> cuda-core[cu13]==0.3.*

Matching this pin that's already there in dask-cuda's test environment:

dask-cuda/dependencies.yaml

Line 162 in c749356

- cuda-core==0.3.*

[jameslamb]

added that in fc467d3

[pentschev]

Thanks @jameslamb for the patience to discuss the numba-cuda/cuda-core/cuda-bindings situation! 😄

[jameslamb]

No problem, thank YOU! I'm happy with the end state we arrived at.

[jameslamb]

/merge