Skip to content

Add CUDA 11.8 migrator#4834

Merged
jakirkham merged 5 commits into
conda-forge:mainfrom
jakirkham-feedstocks:add_cuda_118_migr
Sep 1, 2023
Merged

Add CUDA 11.8 migrator#4834
jakirkham merged 5 commits into
conda-forge:mainfrom
jakirkham-feedstocks:add_cuda_118_migr

Conversation

@jakirkham

@jakirkham jakirkham commented Aug 23, 2023

Copy link
Copy Markdown
Member

Adds a migrator to include CUDA 11.8 as discussed in issue ( conda-forge/conda-forge.github.io#1981 ).


Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

Fixes conda-forge/conda-forge.github.io#1981

Closes conda-forge/ucx-split-feedstock#133
Closes conda-forge/nccl-feedstock#96
Closes conda-forge/pycolmap-feedstock#7

@jakirkham jakirkham requested a review from a team as a code owner August 23, 2023 22:07
@conda-forge-webservices

Copy link
Copy Markdown
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@jakirkham

jakirkham commented Aug 23, 2023

Copy link
Copy Markdown
Member Author

Started a couple test PRs using this migration (updated below).

These both work:

This PR was created and builds part way ok. Then runs into a tooling issue outside the scope of this change ( some bug in cross-compiling linking: conda-forge/pycolmap-feedstock#7 (comment) )

@jakirkham

jakirkham commented Aug 23, 2023

Copy link
Copy Markdown
Member Author

It appears that native builds work ok. However CUDA 11.8 cross-compilation support is absent atm. Started PR ( conda-forge/conda-forge-ci-setup-feedstock#261 ) to add CUDA 11.8 cross-compilation support

Edit: Marking as draft until CUDA 11.8 cross-compilation support is added

@jakirkham

Copy link
Copy Markdown
Member Author

Thoughts on this @conda-forge/core ?

@h-vetinari h-vetinari left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically LGTM!

Comment thread recipe/migrations/cuda118.yaml Outdated
Comment thread recipe/migrations/cuda118.yaml Outdated
Comment on lines +36 to +38
- quay.io/condaforge/linux-anvil-cuda:9.2 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:10.0 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:10.1 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: Do we still need these? I thought they are not zippable because cuda_compiler_version does not list them?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They didn't seem to cause any issues when applying the migration to different feedstocks

Zipping isn't relevant here as this is the ordering section and is only used to preserve the order of these when they do appear

Comment thread recipe/migrations/cuda118.yaml Outdated
Comment on lines +61 to +62
# non-CUDA arch: cross-compilation (build != target)
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: Wouldn't this last item be zipped with cuda_compiler_version == 12.0? (I guess I never understood the meaning of ordering in the migrator, even after the CUDA 12 migration. Would be nice if someone could shed some light 🙂)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(My very same question from the old days: #4400 (comment))

@jakirkham jakirkham Sep 1, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied this to feedstocks that completed the CUDA 12 migration (so they had both migrators like NCCL). AFAICT it worked without issues

This is due to conda-build reordering values of keys (so losing the original ordering). As a result we have to reorder things after the fact in conda-smithy

In any event as the long term goal is to eliminate the need for different Docker images (as is already happening with CUDA 12.0). Think the docker_image ordering can eventually go away. So don't think it is worth thinking to hard about

@h-vetinari h-vetinari left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thx!

@jakirkham jakirkham merged commit 91edef5 into conda-forge:main Sep 1, 2023
@jakirkham jakirkham deleted the add_cuda_118_migr branch September 1, 2023 21:43
@jakirkham

Copy link
Copy Markdown
Member Author

Thanks all! 🙏

Going to go ahead and get this started then. We can follow up on anything else separately

@leofang

leofang commented Sep 2, 2023

Copy link
Copy Markdown
Member

I noticed that cusparselt is marked migrated, but it isn't, likely because it has this skip condition

skip: true  # [ppc64le or cuda_compiler_version not in ("11.2", "12.0")]

and the bot can't see any diff before/after applying the migrator. This would likely happen to other feedstocks like cupy too.

@jakirkham

Copy link
Copy Markdown
Member Author

Are there differences in the cusparselt binary needed for the CUDA 11.8 case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bump minimal CUDA version to CUDA 11.8?

3 participants