Add ROCm (AMD GPU) support to studio setup#4583
Conversation
Add GPU backend detection for ROCm alongside existing CUDA support in the llama.cpp build step. When hipcc is found and no CUDA toolchain is present, the build uses GGML_HIP=ON with proper ROCm root resolution via readlink/hipconfig, sets ROCM_PATH and HIP_PATH, uses the upstream-recommended HIPCXX compiler path, and auto-detects the gfx architecture via rocminfo. CUDA detection is unchanged and takes priority on mixed-toolchain hosts.
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request enhances the studio/setup.sh script to include support for AMD ROCm GPUs, in addition to the existing NVIDIA CUDA support. The script now intelligently detects either CUDA or ROCm, configuring the build process accordingly. For ROCm, it sets GGML_HIP and exports necessary environment variables like ROCM_PATH and HIP_PATH, and attempts to detect specific AMD GPU architectures for targeted compilation. A review comment suggests an improvement to ensure that the directory containing hipcc is consistently added to the PATH to guarantee that all related ROCm tools, such as hipconfig and rocminfo, are discoverable during the build process.
| export PATH="$(dirname "$ROCM_HIPCC"):$PATH" | ||
| GPU_BACKEND="rocm" | ||
| fi | ||
| fi |
There was a problem hiding this comment.
The script correctly adds the hipcc directory to PATH when hipcc is found in specific /opt/rocm locations. However, if hipcc is found via command -v hipcc (meaning it's already in the system's PATH), its directory is not explicitly added to PATH by the script. This could lead to hipconfig or rocminfo not being found later if they reside in the same hipcc directory but that directory is not fully covered by the existing PATH for all necessary ROCm tools. Explicitly adding the hipcc directory to PATH after its discovery ensures all related ROCm tools are discoverable for subsequent commands, regardless of how hipcc was initially found.
| fi | |
| # Ensure the directory containing hipcc is in PATH for hipconfig/rocminfo | |
| if [ -n "$ROCM_HIPCC" ]; then | |
| _HIPCC_DIR="$(dirname "$ROCM_HIPCC")" | |
| # Only add if not already in PATH to avoid duplicates | |
| [[ ":$PATH:" != *":$_HIPCC_DIR:"* ]] && export PATH="$_HIPCC_DIR:$PATH" | |
| fi |
|
Superseded by a new PR that preserves the original contributor's git history. |
1 participant
Summary
Adds AMD ROCm GPU detection to the llama.cpp build step in
studio/setup.sh, based on the work in #4390 by @edamamez, rebased onto current main with bug fixes.hipcc(PATH,/opt/rocm/bin,/opt/rocm-*/bin) only when no CUDA toolchain is presentreadlink -fon hipcc andhipconfig -R(fixes the symlink/variable-collision bug from Add support for ROCm in Studio setup #4390)ROCM_PATHandHIP_PATHfor cmakeHIPCXXviahipconfig -linstead of legacyCMAKE_C_COMPILER=hipcc(fixes upstream deprecation warning)rocminfowith tightened regex (gfx[0-9]{3,4})Changes are limited to the GPU detection block in
studio/setup.sh(71 lines added, 4 lines modified).Bugs fixed vs #4390
ROCM_PATHto the hipcc binary path instead of the ROCm root directory-DCMAKE_C_COMPILER=hipccwhich is deprecated upstreamHIP_PATHexported alongsideROCM_PATHTest plan
git diff origin/maintouches onlystudio/setup.sh