Skip to content

Relax libcufile dependency for old CTK 12 on ARM #930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rapids-bot merged 13 commits into from
Sep 18, 2025
Merged

Relax libcufile dependency for old CTK 12 on ARM #930

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
b0e17af
Relax `libcufile` dependency for old CTK 12 on ARM
jakirkham Aug 22, 2025
f5f3ae9
Simply use `cufile.h`
jakirkham Aug 22, 2025
584e88c
Require cuFile component in GDS stub
jakirkham Aug 22, 2025
f6f2d7d
Merge branch 'branch-25.10' into fix_dep_cufile_arm
jameslamb Sep 8, 2025
4dbfd5a
Update ci/build_cpp.sh
grlee77 Sep 9, 2025
2063e7c
Always build with `libcufile-dev`
jakirkham Sep 18, 2025
61ab317
Ignore `run_exports` of `libcufile-dev` on ARM too
jakirkham Sep 18, 2025
e447f6b
Revert "Update ci/build_cpp.sh"
jakirkham Sep 18, 2025
9dd7bff
Tweak comment about selectively dropping cuFile
jakirkham Sep 18, 2025
99975e2
Align selectors
jakirkham Sep 18, 2025
73dd613
Show what `extra_variants.yaml` contains in CI log
jakirkham Sep 18, 2025
f3fa2b1
Include package hash in cuCIM's build strings
jakirkham Sep 18, 2025
5dbbb43
Note ARM CUDA 12.2+ requires cuFile
jakirkham Sep 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion ci/build_cpp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,26 @@ conda config --set path_conflict warn

sccache --zero-stats

RAPIDS_PACKAGE_VERSION=$(rapids-generate-version) rapids-conda-retry build conda/recipes/libcucim

# TODO: Remove when CUDA 12.1 is dropped.
# In most cases, the CTK has cuFile.
# However the CTK only added cuFile for ARM in 12.2.
# So for users installing with CTK 12.0 & 12.1, relax the cuFile requirement.
# On x86 or CTK 13, always require cuFile.
if [[ "$(arch)" == "aarch64" ]] && [[ "${RAPIDS_CUDA_VERSION%%.*}" == "12" ]]; then
cat > extra_variants.yaml <<EOF
has_cufile:
- False
EOF
else
cat > extra_variants.yaml <<EOF
has_cufile:
- True
EOF
fi
Comment on lines +30 to +39

@grlee77 grlee77 Sep 9, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the current logic will have both in the aarch64+CUDA12 case?

has_cufile:
  - True
  -  False
Suggested change
cat > extra_variants.yaml <<EOF
has_cufile:
- True
EOF
if [[ "$(arch)" == "aarch64" ]] && [[ "${RAPIDS_CUDA_VERSION%%.*}" == "12" ]]
then
cat >> extra_variants.yaml <<EOF
- False
EOF
fi
if [[ "$(arch)" == "aarch64" ]] && [[ "${RAPIDS_CUDA_VERSION%%.*}" == "12" ]]; then
cat > extra_variants.yaml <<EOF
has_cufile:
- False
EOF
else
cat > extra_variants.yaml <<EOF
has_cufile:
- True
EOF
fi

@grlee77 grlee77 Sep 9, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me commit this and see if it helps

@jakirkham jakirkham Sep 18, 2025 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this is intended. We want to build once for each of these values, which is what the original logic does

Also for CUDA 13+ or x86_64 builds we want this to only be True

The CI failure is due to something else unrelated to this syntax. Please see this comment: #930 (comment)

Have gone ahead and restored the original logic. Though happy to chat more if you have additional questions 🙂


RAPIDS_PACKAGE_VERSION=$(rapids-generate-version) rapids-conda-retry build \
-m extra_variants.yaml \
conda/recipes/libcucim

sccache --show-adv-stats
14 changes: 13 additions & 1 deletion conda/recipes/libcucim/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,14 +55,26 @@ requirements:
host:
- cuda-version ={{ cuda_version }}
- cuda-cudart-dev
{% if aarch64 and cuda_major == "12" %}
# Skip building with `libcufile-dev` if it cannot be used at runtime
- libcufile-dev # [has_cufile]
{% else %}
- libcufile-dev
{% endif %}
Comment thread
jakirkham marked this conversation as resolved.
Outdated
- libnvjpeg-dev
- nvtx-c >=3.1.0
- openslide
run:
{% if aarch64 and cuda_major == "12" %}
# Relax `libcufile` dependency for old CUDA 12 ARM installs
- {{ pin_compatible('cuda-version', lower_bound='12.0', upper_bound='12.2') }} # [not has_cufile]
- {{ pin_compatible('cuda-version', lower_bound='12.2', upper_bound='13.0') }} # [has_cufile]
- libcufile # [has_cufile]
{% else %}
Comment on lines +63 to +68

@jakirkham jakirkham Sep 18, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ensures ARM packages that support CUDA 12.2+ still depend io cuFile whereas packages that require CUDA 12.0 or 12.1 packages on ARM don't require cuFile (as it is not given in this case)

- {{ pin_compatible('cuda-version', max_pin='x', min_pin='x') }}
- cuda-cudart
- libcufile
{% endif %}
- cuda-cudart
- libnvjpeg
run_constrained:
- {{ pin_compatible('openslide') }}
Expand Down
2 changes: 1 addition & 1 deletion gds/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ if (NOT APPLE)
endif ()

# Find CUDA Toolkit for cudart and cufile.
find_package(CUDAToolkit REQUIRED)
find_package(CUDAToolkit REQUIRED COMPONENTS cuFile)

################################################################################
# Add library: cufile_stub
Expand Down
7 changes: 1 addition & 6 deletions gds/include/cufile_stub.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,7 @@
#ifndef CUCIM_CUFILE_STUB_H
#define CUCIM_CUFILE_STUB_H

// Try to include the real cufile.h, fall back to minimal types if not available
#if __has_include(<cufile.h>)
#include <cufile.h>
#else
#include "cufile_stub_types.h"
#endif
#include <cufile.h>

@jakirkham jakirkham Aug 22, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to #include "cufile_stub_types.h" causes the following error on CI:

In file included from /opt/conda/conda-bld/work/gds/src/cufile_stub.cpp:17:
/opt/conda/conda-bld/work/gds/include/cufile_stub.h:23:14: fatal error: cufile_stub_types.h: No such file or directory
   23 |     #include "cufile_stub_types.h"
      |              ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.

Given we either have a new enough CUDA Toolkit and it has cuFile or it doesn't, we can simply check for cuFile and build or not

Unsure what we gain by having this in-between mode. Though would be happy to learn if I'm missing something


#include "cucim/dynlib/helper.h"

Expand Down
Loading