Skip to content

fix: [rocSHMEM] Drop LLC dependency in HSACO object compilation (#7817) - #8170

Merged
marbre merged 1 commit into
release/therock-7.14from
users/marbre/7.14-cherry-pick-e707019
Jul 6, 2026
Merged

marbre merged 1 commit into
release/therock-7.14from
users/marbre/7.14-cherry-pick-e707019

Conversation

@marbre

@marbre marbre commented Jul 6, 2026 •

Copy link
Copy Markdown
Member

Cherry-picks e707019 into the 7.14 release branch.


Motivation

JIRA ID: AIROCSHMEM-439

  1. llc is not shipped by all ROCm/LLVM distributions (absent on LLVM 23+), causing the device_bitcode_tester build to be silently skipped at configure time on affected machines.
  2. The HSACO amdhsa.target metadata should include GPU feature suffixes (e.g. gfx950:sramecc+:xnack-) to match what the runtime GPU reports. Previously there was no way to supply these suffixes without a build-time rocminfo call against the local GPU, which fails on GPU-less build hosts and on multi-arch builds where not all target GPUs are physically present. Now, we are letting the user pass those features through GPU_TARGET var to cmake scripts.

Technical Details

1. Replace llc with clang -x ir

  • Replace the llc -filetype=obj step in CMakeDeviceBitcodeTester.cmake with clang -x ir -c -target amdgcn-amd-amdhsa, which performs the same BC→object compilation using clang, which is always present.
  • Arch feature flags (sramecc, xnack, etc.) previously passed as -mattr= flags to llc are now passed as -Xclang -target-feature flags to clang. The helper function arch_features_to_mattr_flags is renamed to arch_features_to_target_feature_flags and updated accordingly.
  • Remove find_program(LLVM_LLC ...) and its guard. llc is no longer a build dependency.
  • llc/ld.lld/opt search paths are extended to also look under ${THEROCK_TOOLCHAIN_ROOT}/lib/llvm/bin, consistent with how clang++ and llvm-link are already searched in DeviceBitcode.cmake.

2. Feature suffix support via GPU_TARGETS

  • Users can now pass feature suffixes directly in the GPU_TARGETS CMake variable:
 ../scripts/build_configs/ipc_single
-DGPU_TARGETS="gfx950:sramecc+:xnack-"
  • The main CMakeLists.txt normalises GPU_TARGETS into bare arch names (required by HIP compilation and rocm_check_target_ids) while preserving the original suffixed strings in a new GPU_TARGETS_FULL cache variable. DeviceBitcode.cmake reads GPU_TARGETS_FULL to populate BITCODE_GPU_ARCHS_FULL, which drives the -Xclang -target-feature flags passed to clang when building the tester HSACOs.
  • The rocminfo dependency for feature suffix resolution is removed entirely. Feature suffixes now come from the user-supplied GPU_TARGETS value.
  • A bug in the original CMakeLists.txt is also fixed: rocm_check_target_ids was always called with DEFAULT_GPU_TARGETS regardless of whether the user supplied GPU_TARGETS, causing the user's value to be silently ignored.

Test Result

  • Functional tests pass on gfx942 and gfx950.
  • Regarding AIROCSHMEM-437, the device bitcode tester is fixed with this PR, but the all-to-all issue on gfx942 should be further investigated (not related to this PR).

JIRA ID: AIROCSHMEM-439

## Motivation

JIRA ID: AIROCSHMEM-439

1. `llc` is not shipped by all ROCm/LLVM distributions (absent on LLVM
23+), causing the `device_bitcode_tester` build to be silently skipped
at configure time on affected machines.
2. The HSACO `amdhsa.target` metadata should include GPU feature
suffixes (e.g. `gfx950:sramecc+:xnack-`) to match what the runtime GPU
reports. Previously there was no way to supply these suffixes without a
build-time `rocminfo` call against the local GPU, which fails on
GPU-less build hosts and on multi-arch builds where not all target GPUs
are physically present. Now, we are letting the user pass those features
through `GPU_TARGET` var to cmake scripts.

## Technical Details

### 1. Replace `llc` with `clang -x ir`

- Replace the `llc -filetype=obj` step in
`CMakeDeviceBitcodeTester.cmake` with `clang -x ir -c -target
amdgcn-amd-amdhsa`, which performs the same BC→object compilation using
`clang`, which is always present.
- Arch feature flags (`sramecc`, `xnack`, etc.) previously passed as
`-mattr=` flags to `llc` are now passed as `-Xclang -target-feature`
flags to `clang`. The helper function `arch_features_to_mattr_flags` is
renamed to `arch_features_to_target_feature_flags` and updated
accordingly.
- Remove `find_program(LLVM_LLC ...)` and its guard. `llc` is no longer
a build dependency.
- `llc`/`ld.lld`/`opt` search paths are extended to also look under
`${THEROCK_TOOLCHAIN_ROOT}/lib/llvm/bin`, consistent with how `clang++`
and `llvm-link` are already searched in `DeviceBitcode.cmake`.

### 2. Feature suffix support via `GPU_TARGETS`

- Users can now pass feature suffixes directly in the `GPU_TARGETS`
CMake variable:

  ```bash
../scripts/build_configs/ipc_single
-DGPU_TARGETS="gfx950:sramecc+:xnack-"
  ```

- The main `CMakeLists.txt` normalises `GPU_TARGETS` into bare arch
names (required by HIP compilation and `rocm_check_target_ids`) while
preserving the original suffixed strings in a new `GPU_TARGETS_FULL`
cache variable. `DeviceBitcode.cmake` reads `GPU_TARGETS_FULL` to
populate `BITCODE_GPU_ARCHS_FULL`, which drives the `-Xclang
-target-feature` flags passed to `clang` when building the tester
HSACOs.
- The `rocminfo` dependency for feature suffix resolution is removed
entirely. Feature suffixes now come from the user-supplied `GPU_TARGETS`
value.
- A bug in the original `CMakeLists.txt` is also fixed:
`rocm_check_target_ids` was always called with `DEFAULT_GPU_TARGETS`
regardless of whether the user supplied `GPU_TARGETS`, causing the
user's value to be silently ignored.

## Test Result

- Functional tests pass on `gfx942` and `gfx950`.
- Regarding AIROCSHMEM-437, the device bitcode tester is fixed with this
PR, but the all-to-all issue on `gfx942` should be further investigated
(not related to this PR).

---
JIRA ID: AIROCSHMEM-439
@marbre
marbre requested review from a team July 6, 2026 20:52
@marbre
marbre requested a review from a team as a code owner July 6, 2026 20:52
@github-actions github-actions Bot added documentation Improvements or additions to documentation project: rocshmem labels Jul 6, 2026
@marbre
marbre merged commit 10de689 into release/therock-7.14 Jul 6, 2026
14 checks passed
@marbre
marbre deleted the users/marbre/7.14-cherry-pick-e707019 branch July 6, 2026 22:02
marbre added a commit to ROCm/TheRock that referenced this pull request Jul 7, 2026
Bumps rocm-libraries and rocm-systems to include the following
cherry-picks:

* ROCm/rocm-libraries#9105
* ROCm/rocm-libraries#9110
* ROCm/rocm-systems#8170
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation organization: ROCm project: rocshmem

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants