Skip to content

Add ROCm (AMD GPU) support to studio setup #4583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
danielhanchen wants to merge 1 commit into from
Closed

Add ROCm (AMD GPU) support to studio setup #4583

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 71 additions & 4 deletions studio/setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -449,20 +449,43 @@ rm -rf "$LLAMA_CPP_DIR"
echo " Using ccache for faster compilation"
fi

# Detect CUDA: check nvcc on PATH, then common install locations
# Detect GPU backend: CUDA (NVIDIA) or ROCm (AMD)
GPU_BACKEND=""

# Check for CUDA: check nvcc on PATH, then common install locations
NVCC_PATH=""
if command -v nvcc &>/dev/null; then
NVCC_PATH="$(command -v nvcc)"
GPU_BACKEND="cuda"
elif [ -x /usr/local/cuda/bin/nvcc ]; then
NVCC_PATH="/usr/local/cuda/bin/nvcc"
export PATH="/usr/local/cuda/bin:$PATH"
GPU_BACKEND="cuda"
elif ls /usr/local/cuda-*/bin/nvcc &>/dev/null 2>&1; then
# Pick the newest cuda-XX.X directory
NVCC_PATH="$(ls -d /usr/local/cuda-*/bin/nvcc 2>/dev/null | sort -V | tail -1)"
export PATH="$(dirname "$NVCC_PATH"):$PATH"
GPU_BACKEND="cuda"
fi

# Check for ROCm (AMD) only if CUDA was not already selected
ROCM_HIPCC=""
if [ -z "$GPU_BACKEND" ]; then
if command -v hipcc &>/dev/null; then
ROCM_HIPCC="$(command -v hipcc)"
GPU_BACKEND="rocm"
elif [ -x /opt/rocm/bin/hipcc ]; then
ROCM_HIPCC="/opt/rocm/bin/hipcc"
export PATH="/opt/rocm/bin:$PATH"
GPU_BACKEND="rocm"
elif ls /opt/rocm-*/bin/hipcc &>/dev/null 2>&1; then
ROCM_HIPCC="$(ls -d /opt/rocm-*/bin/hipcc 2>/dev/null | sort -V | tail -1)"
export PATH="$(dirname "$ROCM_HIPCC"):$PATH"
GPU_BACKEND="rocm"
fi
fi
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The script correctly adds the hipcc directory to PATH when hipcc is found in specific /opt/rocm locations. However, if hipcc is found via command -v hipcc (meaning it's already in the system's PATH), its directory is not explicitly added to PATH by the script. This could lead to hipconfig or rocminfo not being found later if they reside in the same hipcc directory but that directory is not fully covered by the existing PATH for all necessary ROCm tools. Explicitly adding the hipcc directory to PATH after its discovery ensures all related ROCm tools are discoverable for subsequent commands, regardless of how hipcc was initially found.

Suggested change
fi
# Ensure the directory containing hipcc is in PATH for hipconfig/rocminfo
if [ -n "$ROCM_HIPCC" ]; then
_HIPCC_DIR="$(dirname "$ROCM_HIPCC")"
# Only add if not already in PATH to avoid duplicates
[[ ":$PATH:" != *":$_HIPCC_DIR:"* ]] && export PATH="$_HIPCC_DIR:$PATH"
fi


if [ -n "$NVCC_PATH" ]; then
if [ "$GPU_BACKEND" = "cuda" ]; then
echo " Building with CUDA support (nvcc: $NVCC_PATH)..."
CMAKE_ARGS="$CMAKE_ARGS -DGGML_CUDA=ON"

Expand Down Expand Up @@ -494,11 +517,55 @@ rm -rf "$LLAMA_CPP_DIR"

# Multi-threaded nvcc compilation (uses all CPU cores per .cu file)
CMAKE_ARGS="$CMAKE_ARGS -DCMAKE_CUDA_FLAGS=--threads=0"
elif [ -d /usr/local/cuda ] || nvidia-smi &>/dev/null; then
elif [ "$GPU_BACKEND" = "rocm" ]; then
# Resolve hipcc symlinks to find the real ROCm root
_HIPCC_REAL="$(readlink -f "$ROCM_HIPCC" 2>/dev/null || printf '%s' "$ROCM_HIPCC")"
ROCM_ROOT=""
if command -v hipconfig &>/dev/null; then
ROCM_ROOT="$(hipconfig -R 2>/dev/null || true)"
fi
if [ -z "$ROCM_ROOT" ]; then
ROCM_ROOT="$(cd "$(dirname "$_HIPCC_REAL")/.." 2>/dev/null && pwd)"
fi

echo " Building with ROCm support (AMD GPU, hipcc: $_HIPCC_REAL)..."
CMAKE_ARGS="$CMAKE_ARGS -DGGML_HIP=ON"
export ROCM_PATH="$ROCM_ROOT"
export HIP_PATH="$ROCM_ROOT"

# Use upstream-recommended HIP compiler (not legacy hipcc-as-CXX)
if command -v hipconfig &>/dev/null; then
_HIP_CLANG_DIR="$(hipconfig -l 2>/dev/null || true)"
[ -n "$_HIP_CLANG_DIR" ] && export HIPCXX="$_HIP_CLANG_DIR/clang"
fi

# Detect AMD GPU architecture (gfx target)
GPU_TARGETS=""
if command -v rocminfo &>/dev/null; then
_gfx_list=$(rocminfo 2>/dev/null | grep -oE 'gfx[0-9]{3,4}[a-z]?' | sort -u || true)
_valid_gfx=""
for _gfx in $_gfx_list; do
if [[ "$_gfx" =~ ^gfx[0-9]{3,4}[a-z]?$ ]]; then
_valid_gfx="${_valid_gfx}${_valid_gfx:+;}$_gfx"
fi
done
[ -n "$_valid_gfx" ] && GPU_TARGETS="$_valid_gfx"
fi

if [ -n "$GPU_TARGETS" ]; then
echo " AMD GPU architectures: ${GPU_TARGETS//;/, } -- limiting build to detected targets"
CMAKE_ARGS="$CMAKE_ARGS -DGPU_TARGETS=${GPU_TARGETS}"
else
echo " Could not detect AMD GPU arch -- building for default targets (cmake will auto-detect)"
fi
elif [ -d /usr/local/cuda ] || command -v nvidia-smi &>/dev/null; then
echo " CUDA driver detected but nvcc not found — building CPU-only"
echo " To enable GPU: install cuda-toolkit or add nvcc to PATH"
elif [ -d /opt/rocm ] || command -v rocm-smi &>/dev/null; then
echo " ROCm driver detected but hipcc not found — building CPU-only"
echo " To enable GPU: install rocm-dev or add hipcc to PATH"
else
echo " Building CPU-only (no CUDA detected)..."
echo " Building CPU-only (no CUDA or ROCm detected)..."
fi

NCPU=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)
Expand Down
Loading