Skip to content

Rebuild for flang 19#53

Merged
h-vetinari merged 11 commits into
conda-forge:mainfrom
regro-cf-autotick-bot:rebuild-flang19-0-2_hc30e88
Feb 26, 2025
Merged

Rebuild for flang 19#53
h-vetinari merged 11 commits into
conda-forge:mainfrom
regro-cf-autotick-bot:rebuild-flang19-0-2_hc30e88

Conversation

@regro-cf-autotick-bot

Copy link
Copy Markdown
Contributor

This PR has been triggered in an effort to update flang19.

Notes and instructions for merging this PR:

  1. Please merge the PR only after the tests have passed.
  2. Feel free to push to the bot's branch to update this PR if needed.

Please note that if you close this PR we presume that the feedstock has been rebuilt, so if you are going to perform the rebuild yourself don't close this PR until the your rebuild has been merged.


Here are some more details about this specific migrator:

TL;DR: We are trying to switch our Fortran compilers on windows to flang.
This is not 100% guaranteed to work, but should be fine in the majority of cases.

The new LLVM-based flang has become mature enough that it should be possible to
broadly switch over our Fortran compilers on windows to it (until now we only had
an ancient pre-LLVM flang 5, or alternatively the GCC-based m2w64_fortran).

As such, this PR attempts to homogenize any use of m2w64_fortran and other m2w64_*
compilers to our default stack (which would then be MSVC + flang on windows), with
the exception of feedstocks for R-packages, which stay on the m2w64_ compilers.

Recipes that have hard-coded expectations about the name of the fortran compiler
will need to adjust to use %FC% or flang-new for the compiler name. Similarly,
you may need to change the linker to %LD% or use lld-link.

It is also possible that you run into compilation errors due to differences in
compiler behaviour, bugs or as-yet unimplemented features. In case of compilation
errors due to stricter default language standards, you should be able to fix things
by passing -std=legacy to FFLAGS.

If you have problems with this PR, feel free to ping the @c-f/flang-activation team.
In case you have convinced yourself that flang really is not ready yet to be used to
compile a given feedstock, you may also close this migrator PR.


If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/12160780225 - please use this URL for debugging.

TL;DR: We are trying to switch our Fortran compilers on windows to flang.
This is not 100% guaranteed to work, but should be fine in the majority of cases.

The new LLVM-based flang has become mature enough that it should be possible to
broadly switch over our Fortran compilers on windows to it (until now we only had
an ancient pre-LLVM flang 5, or alternatively the GCC-based `m2w64_fortran`).

As such, this PR attempts to homogenize any use of `m2w64_fortran` and other `m2w64_*`
compilers to our default stack (which would then be MSVC + flang on windows), with
the exception of feedstocks for R-packages, which stay on the `m2w64_` compilers.

Recipes that have hard-coded expectations about the name of the fortran compiler
will need to adjust to use `%FC%` or `flang-new` for the compiler name. Similarly,
you may need to change the linker to `%LD%` or use `lld-link`.

It is also possible that you run into compilation errors due to differences in
compiler behaviour, bugs or as-yet unimplemented features. In case of compilation
errors due to stricter default language standards, you should be able to fix things
by passing `-std=legacy` to `FFLAGS`.

If you have problems with this PR, feel free to ping the @c-f/flang-activation team.
In case you have convinced yourself that flang really is not ready yet to be used to
compile a given feedstock, you may also close this migrator PR.
@conda-forge-admin

conda-forge-admin commented Dec 4, 2024

Copy link
Copy Markdown
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

  • ℹ️ The recipe is not parsable by parser conda-souschef (grayskull). This parser is not currently used by conda-forge, but may be in the future. We are collecting information to see which recipes are compatible with grayskull.
  • ℹ️ The recipe is not parsable by parser conda-recipe-manager. The recipe can only be automatically migrated to the new v1 format if it is parseable by conda-recipe-manager.

This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/13540757631. Examine the logs at this URL for more detail.

@h-vetinari

Copy link
Copy Markdown
Member

@conda-forge-admin, please rerender

conda-forge-webservices[bot] and others added 2 commits January 24, 2025 21:05
@h-vetinari

Copy link
Copy Markdown
Member

@jakirkham, this is one of the places where the automatic CUDA 12.0 -> 12.6 upgrade failed. We're getting

$PREFIX/targets/x86_64-linux/include/nvToolsExt.h(248): error: invalid redeclaration of type name "nvtxStringHandle_t" (declared at line 292 of $PREFIX/targets/x86_64-linux/include/nvtx3/nvToolsExt.h)
  typedef struct nvtxStringHandle* nvtxStringHandle_t;
                                   ^

Both of these header files are in the same package (cuda-nvtx-dev), so I can't even remove one.

@h-vetinari

h-vetinari commented Jan 24, 2025

Copy link
Copy Markdown
Member

The workaround on windows that John had recommended (eb88dfc) to use nvtx-c instead of cuda-nvtx-dev also doesn't work:

-- Detecting CUDA compile features - done
CMake Error at /home/conda/feedstock_root/build_artifacts/kaldi-split_1737759691433/_build_env/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
  Could NOT find NvToolExt (missing: NvToolExt_INCLUDE_DIR
  NvToolExt_LIBRARIES)

However, I did notice belatedly that the PPC build with 12.4 had passed before, meaning that we can narrow down the behaviour change or breakage from (12.0, 12.6] to (12.4, 12.6]. CC @conda-forge/cuda-nvtx @conda-forge/cuda

@bdice

bdice commented Feb 4, 2025

Copy link
Copy Markdown

I don't know much about this feedstock, but I can add that nvtx-c ships only headers like include/nvToolsExt.h and include/nvtx3.hpp. It does not include any CMake configuration files (but it probably should), and it does not put files into the CUDA targets layout like targets/x86_64-linux/include/ -- this is intentional, as nvtx-c intends to be a package that is usable independent of other CUDA bits.

@h-vetinari

Copy link
Copy Markdown
Member

Thanks for the response! nvtx-c was the workaround for the failure, ideally we'd keep using cuda-nvtx-dev. However, that fails with

$PREFIX/targets/x86_64-linux/include/nvToolsExt.h(248): error: invalid redeclaration of type name "nvtxStringHandle_t" (declared at line 292 of $PREFIX/targets/x86_64-linux/include/nvtx3/nvToolsExt.h)
  typedef struct nvtxStringHandle* nvtxStringHandle_t;
                                   ^

@bdice

bdice commented Feb 4, 2025

Copy link
Copy Markdown

I believe this code may be using NVTX v2 and v3 at the same time, which may not be supported. Do we know where these includes are coming from? Ideally we would use only $PREFIX/targets/x86_64-linux/include/nvtx3/nvToolsExt.h.

@h-vetinari

Copy link
Copy Markdown
Member

Thanks for the input @bdice!

@mmcauliffe, could I perhaps convince you to take a look at the CUDA 12.6 / nvtx situation here? I believe it's the lack of the rebuild for 12.6 that's behind the segfaults observed in torchaudio: conda-forge/torchaudio-feedstock#15

@h-vetinari h-vetinari force-pushed the rebuild-flang19-0-2_hc30e88 branch from b67f663 to a87de7a Compare February 26, 2025 06:20
@h-vetinari h-vetinari force-pushed the rebuild-flang19-0-2_hc30e88 branch from 593fd81 to e1dec54 Compare February 26, 2025 08:51
@h-vetinari h-vetinari mentioned this pull request Feb 26, 2025
@h-vetinari h-vetinari merged commit 109e212 into conda-forge:main Feb 26, 2025
@regro-cf-autotick-bot regro-cf-autotick-bot deleted the rebuild-flang19-0-2_hc30e88 branch February 26, 2025 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants