Upgrade to nvcomp 5.3.0.16

[bdice]

Description

Upgrades nvcomp to version 5.3.0.16.

Depends on rapidsai/rmm#2462.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[robertmaynard]

Approving CMake changes

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

NVComp pins are updated to 5.3.0.16 across dependency matrices, Python packaging, build configuration, and CUDA environment definitions.

Changes

NVComp dependency update

Layer / File(s)	Summary
Dependency matrix and Python pins dependencies.yaml, python/libcudf/pyproject.toml	Updates conda and Python NVComp pins to 5.3.0.16.
Build configuration update conda/recipes/libcudf/conda_build_config.yaml, cpp/cmake/thirdparty/get_nvcomp.cmake	Updates the conda and CMake NVComp version, along with SPDX header wording.
CUDA environment pins conda/environments/all_cuda-129_*.yaml, conda/environments/all_cuda-133_*.yaml	Updates libnvcomp-dev pins for CUDA 12.9 and 13.3 across aarch64 and x86_64 environments.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested labels: improvement

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: upgrading nvcomp to 5.3.0.16.
Description check	✅ Passed	The description is directly related and accurately states the nvcomp upgrade and its dependency on the RMM PR.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[bdice]

@robertmaynard Could I get your review of these final changes I made after your last commit? 8fead2a...5a80603

[robertmaynard]

nvcomp_static already whole-archives rmm and rapids_logger

There is no promise that nvcomp is going to whole archive the same version of rmm that cudf has built against. We need to whole archive rmm even when nvcomp_static is being used

[bdice]

This was changed in 5a80603. Prior to that, CI failed in the linters job: https://github.com/rapidsai/cudf/actions/runs/30135081055/job/89617508579

Errors look like:

/opt/conda/envs/clang_tidy/bin/x86_64-conda-linux-gnu-ld: _deps/rapids_logger-build/librapids_logger.a(logger.cpp.o): in function `rapids_logger::detail::default_flush()':
logger.cpp:(.text._ZN13rapids_logger6detail13default_flushEv+0x0): multiple definition of `rapids_logger::detail::default_flush()'; _deps/nvcomp_proprietary_binary-src/lib/libnvcomp_static.a(logger.cpp.o):logger.cpp:(.text+0x0): first defined here
/opt/conda/envs/clang_tidy/bin/x86_64-conda-linux-gnu-ld: _deps/rapids_logger-build/librapids_logger.a(logger.cpp.o): in function `rapids_logger::sink::sink(std::unique_ptr<rapids_logger::detail::sink_impl, std::default_delete<rapids_logger::detail::sink_impl> >)':
logger.cpp:(.text._ZN13rapids_logger4sinkC2ESt10unique_ptrINS_6detail9sink_implESt14default_deleteIS3_EE+0x0): multiple definition of `rapids_logger::sink::sink(std::unique_ptr<rapids_logger::detail::sink_impl, std::default_delete<rapids_logger::detail::sink_impl> >)'; _deps/nvcomp_proprietary_binary-src/lib/libnvcomp_static.a(logger.cpp.o):logger.cpp:(.text+0x2e0): first defined here

[robertmaynard]

Yeah you have to disable or ignore those linter checks. We know we have duplicate symbols in our archives since rapids-logger is being embedded into nvcomp. We don't have a way to avoid that currently and are relying on the fact that rapids-logger is ABI stable.

[robertmaynard]

Why was spdlog different compared to rmm and rapids-logger?

[bdice]

f404547 was needed because of this issue where fmt was being exported as a dependency. This made the cuDF CMake tests fail: https://github.com/rapidsai/cudf/actions/runs/30124560885/job/89585363258#step:13:1511

 -- Found cudf: /__w/cudf/cudf/cpp/build/cmake-tests/cudf_export_consumer/cudf-build/cudf-config.cmake (found version "26.08.0")
CMake Error at CMakeLists.txt:16 (message):
  cudf::cudf INTERFACE_LINK_LIBRARIES exposes private dependency fmt:::
  CCCL::CCCL;dl;CUDA::cudart_static;Threads::Threads;fmt::fmt-header-only;CCCL::CCCL


-- Configuring incomplete, errors occurred!
ninja: build stopped: subcommand failed.

I don't know if this is the right fix, or if the test itself is flawed.

[robertmaynard]

Okay that makes sense, rapids-logger and rmm are part of the public API and have to be exported

[bdice]

Replaced by #23456.