Update ROCm to 7.14.0 for Docker builds

.devops/rocm.Dockerfile

@@ -1,11 +1,11 @@
-ARG UBUNTU_VERSION=24.04
+ARG UBUNTU_VERSION=26.04
 
 # This needs to generally match the container host's environment.
-ARG ROCM_VERSION=7.2.1
-ARG AMDGPU_VERSION=7.2.1
+ARG ROCM_VERSION=7.14.0
+ARG AMDGPU_VERSION=7.14.0
 
 # Target the ROCm build image
-ARG BASE_ROCM_DEV_CONTAINER=docker.io/rocm/dev-ubuntu-${UBUNTU_VERSION}:${ROCM_VERSION}-complete
+ARG BASE_ROCM_DEV_CONTAINER=docker.io/rocm/dev-ubuntu-${UBUNTU_VERSION}:${ROCM_VERSION}-full
 
 ARG BUILD_DATE=N/A
 ARG APP_VERSION=N/A
@@ -30,11 +30,16 @@ FROM ${BASE_ROCM_DEV_CONTAINER} AS build
 
 # Unless otherwise specified, we make a fat build.
 # This is mostly tied to rocBLAS supported archs.
-# check https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.1/reference/system-requirements.html
-# check https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/docs/compatibility/compatibilityrad/native_linux/native_linux_compatibility.html
-# check https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/docs/compatibility/compatibilityryz/native_linux/native_linux_compatibility.html
+# check https://rocm.docs.amd.com/en/docs-7.14.0/install/rocm.html
+# check https://rocm.docs.amd.com/en/docs-7.14.0/reference/gpu-specs.html
 
-ARG ROCM_DOCKER_ARCH='gfx908;gfx90a;gfx942;gfx1030;gfx1100;gfx1101;gfx1102;gfx1151;gfx1150;gfx1200;gfx1201'
+# Discrete GPUs plus the fastest APUs, out of the architectures that have an
+# amdrocm-blas7.14-gfx*** package inside the container. The remaining APU targets
+# (gfx1033, gfx1035, gfx1036, gfx1103, gfx1152, gfx1153) are left out to keep the
+# build within the CI time budget - they are several times slower than gfx1151
+# anyway. ROCm supports them, so build locally with e.g.
+# --build-arg ROCM_DOCKER_ARCH=gfx1103 if you need one of them.
+ARG ROCM_DOCKER_ARCH='gfx908;gfx90a;gfx942;gfx950;gfx1010;gfx1011;gfx1012;gfx1030;gfx1031;gfx1032;gfx1034;gfx1100;gfx1101;gfx1102;gfx1150;gfx1151;gfx1200;gfx1201'
 
 # Set ROCm architectures
 ENV AMDGPU_TARGETS=${ROCM_DOCKER_ARCH}
@@ -90,8 +95,14 @@ LABEL org.opencontainers.image.created=$BUILD_DATE \
       org.opencontainers.image.url=$IMAGE_URL \
       org.opencontainers.image.source=$IMAGE_SOURCE
 
+# ROCm's amdrocm-* packaging installs the libraries under /opt/rocm/core-<ver>/lib
+# behind /etc/alternatives and no longer registers them with the dynamic loader.
+# Without this, libggml-hip.so cannot resolve libamdhip64/libhipblas/librocblas
+# at runtime and ggml silently skips the HIP backend -> "no usable GPU found".
 RUN apt-get update \
     && apt-get install -y libgomp1 curl ffmpeg \
+    && printf '/opt/rocm/lib\n/opt/rocm/lib/llvm/lib\n' > /etc/ld.so.conf.d/rocm.conf \
+    && ldconfig \
     && apt autoremove -y \
     && apt clean -y \
     && rm -rf /tmp/* /var/tmp/* \

docs/build.md

@@ -389,6 +389,10 @@ You can download it from your Linux distro's package manager or from here: [ROCm
 The environment variable [`HIP_VISIBLE_DEVICES`](https://rocm.docs.amd.com/en/latest/understand/gpu_isolation.html#hip-visible-devices) can be used to specify which GPU(s) will be used.
 If your GPU is not officially supported you can use the environment variable [`HSA_OVERRIDE_GFX_VERSION`] set to a similar GPU, for example 10.3.0 on RDNA2 (e.g. gfx1030, gfx1031, or gfx1035) or 11.0.0 on RDNA3. Note that [`HSA_OVERRIDE_GFX_VERSION`] is [not supported on Windows](https://github.com/ROCm/ROCm/issues/2654)
 
+As of ROCm 7.14.0, AMD ships a separate optimized library package for each supported architecture, covering the CDNA targets and everything from RDNA 1 (`gfx1010`) through RDNA 4 (`gfx1201`). For those architectures `HSA_OVERRIDE_GFX_VERSION` is no longer needed and should be left unset, as long as the binary was compiled for the target — build for your real target instead, since spoofing a different architecture now only costs performance. The override is still required to run a binary that has no code for your GPU, such as a prebuilt package covering a different set of architectures. In particular the whole RDNA 2 generation is covered, not just `gfx1030`, so `gfx1031` through `gfx1036` no longer need the `10.3.0` override shown above. Note that this is wider than AMD's official support matrix: RDNA 1 is not officially supported, and of RDNA 2 only `gfx1030` is — the rest work, but are not tuned to the same degree as RDNA 3 and newer. The full list is in [Docker With ROCm](docker.md#docker-with-rocm), which also applies to local builds — note that the prebuilt Docker images cover only part of it (discrete GPUs plus Strix Halo), while a local build such as the one above can target any of these architectures.
+
+Note that Docker image only covers user space: the `amdgpu` kernel module and the GPU firmware come from the host and have to be recent enough for the ROCm release you build against, whether that is AMD's `amdgpu-dkms` package or an upstream kernel with a current `linux-firmware`.
+
 ### Unified Memory
 
 On Linux it is possible to use unified memory architecture (UMA) to share main memory between the CPU and integrated GPU by setting environment variable `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1`. However, this hurts performance for non-integrated GPUs (but enables working with integrated GPUs).

docs/docker.md

@@ -107,6 +107,120 @@ docker run --gpus all -v /path/to/models:/models local/llama.cpp:light-cuda -m /
 docker run --gpus all -v /path/to/models:/models local/llama.cpp:server-cuda -m /models/7B/ggml-model-q4_0.gguf --port 8080 --host 0.0.0.0 -n 512 --n-gpu-layers 1
 ```
 
+## Docker With ROCm
+
+The ROCm images are built from `rocm/dev-ubuntu-26.04:7.14.0-full` and contain the ROCm 7.14.0 user-space runtime.
+
+The `amdgpu` kernel module and the GPU firmware are **not** part of the image — they always come from the host, and they have to be recent enough for the ROCm version running inside the container. Either:
+
+- install AMD's packages on the host (`amdgpu-install --usecase=dkms`, or the `amdgpu-dkms` package), matching the same ROCm release, or
+- if you would rather not use AMD's repositories, run an upstream kernel new enough to support your GPU together with a current [`linux-firmware`](https://gitlab.com/kernel-firmware/linux-firmware).
+
+A host driver or firmware that is too old typically shows up as the GPU not being listed at all, rather than as an explicit error. Check the host side before involving llama.cpp:
+
+```bash
+docker run --rm \
+    --device /dev/kfd --device /dev/dri \
+    --group-add $(getent group video | cut -d: -f3) \
+    --group-add $(getent group render | cut -d: -f3) \
+    --entrypoint rocminfo ghcr.io/ggml-org/llama.cpp:server-rocm
+```
+
+Note that the group IDs have to be passed numerically: the images do not define `video` and `render` groups, so `--group-add render` fails to resolve.
+
+### Supported GPU architectures
+
+ROCm 7.14.0 ships an optimized library package for every architecture below, but the prebuilt images do not cover all of them: building a fat binary for the full set exceeds the CI time budget, so the images are limited to the discrete GPUs plus the fastest APUs, Strix Point (`gfx1150`) and Strix Halo (`gfx1151`). The smaller APUs are several times slower, and remain supported targets that you build locally — see [Building Docker locally](#building-docker-locally-1).
+
+The "Prebuilt" column says whether a target is included in the published images. Parentheses group SKU variants that share a target, so `RX 6700(XT)` covers both the RX 6700 and the RX 6700 XT.
+
+| Family | Target | Prebuilt | Products |
+| --- | --- | --- | --- |
+| CDNA (server & data center accelerators) | `gfx908` | yes | Instinct MI100 |
+| | `gfx90a` | yes | Instinct MI210, MI250(X) |
+| | `gfx942` | yes | Instinct MI300A, MI300X(-HF), MI308X, MI325X |
+| | `gfx950` | yes | Instinct MI350(X/P), MI355X |
+| RDNA 1 (Navi 1X — consumer & workstation) | `gfx1010` | yes | Radeon RX 5700(XT/M), RX 5600(XT/OEM/M), Pro W5700(X), Pro 5700(XT) |
+| | `gfx1011` | yes | Radeon Pro 5600M, Pro V520, Pro V540 |
+| | `gfx1012` | yes | Radeon RX 5500(XT/OEM/M), RX 5300(XT OEM/M), Pro W5500(X/M), Pro 5500(XT/M), Pro 5300(M) |
+| RDNA 2 (Navi 2X — consumer, workstation & APUs) | `gfx1030` | yes | Radeon RX 6950 XT, RX 6900 XT, RX 6800(XT), Pro W6800(X), Pro W6800X Duo, Pro W6900X, Pro V620 |
+| | `gfx1031` | yes | Radeon RX 6750(XT/GRE), RX 6700(XT/M), RX 6850M XT, RX 6800M, Pro W6700 |
+| | `gfx1032` | yes | Radeon RX 6650(XT/M/M XT), RX 6600(XT/M/S), RX 6800S, RX 6700S, Pro W6600(X/M) |
+| | `gfx1033` | no | Steam Deck LCD/OLED APU — "Van Gogh" ("Aerith", "Sephiroth") |
+| | `gfx1034` | yes | Radeon RX 6500(XT/M), RX 6400, RX 6300(M), RX 6550M, RX 6450M, Pro W6400, Pro W6300(M), Pro W6500M |
+| | `gfx1035` | no | Radeon 680M, 660M, 610M — Ryzen 6000 "Rembrandt"/"Rembrandt-R" mobile APUs |
+| | `gfx1036` | no | Radeon Graphics 2CU — Ryzen 7000/9000 desktop, Threadripper 7000/9000, EPYC 4004/4005; Radeon 610M — "Dragon Range" mobile |
+| RDNA 3 (Navi 3X — consumer, workstation & APUs) | `gfx1100` | yes | Radeon RX 7900(XT/XTX/GRE/M), Pro W7900, Pro W7800 |
+| | `gfx1101` | yes | Radeon RX 7800 XT, RX 7700 XT, Pro W7700, Pro V710 |
+| | `gfx1102` | yes | Radeon RX 7600(XT/S/M/M XT), RX 7650 GRE, RX 7700S, RX 7550M, Pro W7600, Pro W7500 |
+| | `gfx1103` | no | Radeon 780M, 760M, 740M, 610M — Ryzen 7040/8040 "Phoenix/Phoenix 2/Hawk Point" mobile APUs; Ryzen Z1(Extreme), Ryzen Z2 Go |
+| RDNA 3.5 (Ryzen AI 300 / Max — APUs) | `gfx1150` | yes | Radeon 890M, 880M — Ryzen AI 300 "Strix Point" APUs; Ryzen Z2 Extreme |
+| | `gfx1151` | yes | Radeon 8060S, 8050S — Ryzen AI Max "Strix Halo" APUs |
+| | `gfx1152` | no | Radeon 860M, 820M — Ryzen AI "Krackan Point" APUs |
+| | `gfx1153` | no | Radeon 840M — Ryzen AI "Krackan Point 2" APUs |
+| RDNA 4 (Navi 4X — consumer & pro) | `gfx1200` | yes | Radeon RX 9060(XT/M/S), RX 9050 |
+| | `gfx1201` | yes | Radeon RX 9070(XT/GRE/S/M XT), RX 9080M, Radeon AI PRO R9700 |
+
+ROCm 7.14.0 has optimized libraries for every target above, so `HSA_OVERRIDE_GFX_VERSION` is only needed when the image itself has no code for your GPU:
+
+- prebuilt image, `yes` row — leave the override unset;
+- prebuilt image, `no` row — set the override, or better, build locally for your target;
+- local build — leave it unset and pass your target to `ROCM_DOCKER_ARCH`.
+
+Spoofing another architecture costs performance and can give incorrect results, so use it only as a fallback.
+
+This is a wider set than AMD's official support matrix, which is worth keeping in mind:
+
+- RDNA 1 (`gfx1010`, `gfx1011`, `gfx1012`) is not officially supported at all. ROCm still ships packages for it and it does work in practice — an RX 5500M (`gfx1012`) handles Qwen 3.5 4B fine, for instance — but expect it to be validated rather than tuned.
+- For RDNA 2, only `gfx1030` is officially supported. The notable change in 7.14.0 is that the rest of the generation (`gfx1031` through `gfx1036`) now has its own packages too, so those cards no longer need `HSA_OVERRIDE_GFX_VERSION=10.3.0` to masquerade as `gfx1030` — provided llama.cpp was built for the target. `gfx1031`, `gfx1032` and `gfx1034` are in the prebuilt images and work without the override; `gfx1035` and `gfx1036` are not, so on a prebuilt image they still need it, and a local build for the real target is the better option.
+
+Treat everything outside the official matrix as working but less optimized than RDNA 3 and newer, where AMD puts most of the tuning effort.
+
+## Building Docker locally
+
+```bash
+docker build -t local/llama.cpp:full-rocm --target full -f .devops/rocm.Dockerfile .
+docker build -t local/llama.cpp:light-rocm --target light -f .devops/rocm.Dockerfile .
+docker build -t local/llama.cpp:server-rocm --target server -f .devops/rocm.Dockerfile .
+```
+
+`ROCM_DOCKER_ARCH` selects the architectures to compile for. It defaults to the same set as the prebuilt images, so pass it explicitly in either of these cases:
+
+- your GPU is one of the `no` rows in the table above — the APU targets `gfx1033`, `gfx1035`, `gfx1036`, `gfx1103`, `gfx1152` and `gfx1153`;
+- you only care about your own GPU and want a much shorter build and a smaller image.
+
+```bash
+docker build -t local/llama.cpp:server-rocm --target server \
+    --build-arg ROCM_DOCKER_ARCH=gfx1103 -f .devops/rocm.Dockerfile .
+```
+
+Multiple targets are separated by semicolons, for example `--build-arg ROCM_DOCKER_ARCH='gfx1100;gfx1103'`. Build time and image size scale with the number of architectures, which is why the published images stop at the discrete GPUs and Strix Halo.
+
+Refer to [.devops/rocm.Dockerfile](../.devops/rocm.Dockerfile) for the remaining `ARGS` and their defaults.
+
+## Usage
+
+After building locally, usage is similar to the non-ROCm examples, but you'll need to expose the GPU devices to the container and use the `--n-gpu-layers` flag.
+
+```bash
+docker run --device /dev/kfd --device /dev/dri \
+    --group-add $(getent group video | cut -d: -f3) \
+    --group-add $(getent group render | cut -d: -f3) \
+    -v /path/to/models:/models local/llama.cpp:full-rocm --run -m /models/7B/ggml-model-q4_0.gguf -p "Building a website can be done in 10 simple steps:" -n 512 --n-gpu-layers 99
+docker run --device /dev/kfd --device /dev/dri \
+    --group-add $(getent group video | cut -d: -f3) \
+    --group-add $(getent group render | cut -d: -f3) \
+    -v /path/to/models:/models local/llama.cpp:light-rocm -m /models/7B/ggml-model-q4_0.gguf -p "Building a website can be done in 10 simple steps:" -n 512 --n-gpu-layers 99
+docker run --device /dev/kfd --device /dev/dri \
+    --group-add $(getent group video | cut -d: -f3) \
+    --group-add $(getent group render | cut -d: -f3) \
+    -p 8080:8080 -v /path/to/models:/models local/llama.cpp:server-rocm -m /models/7B/ggml-model-q4_0.gguf --port 8080 --host 0.0.0.0 -n 512 --n-gpu-layers 99
+```
+
+*Notes:*
+- `--device /dev/dri` exposes every render node on the host. To restrict the container to one GPU, pass the individual nodes instead, for example `--device /dev/dri/renderD128 --device /dev/dri/card0`.
+- If llama.cpp reports `no usable GPU found` while `rocminfo` inside the same container does list the GPU, the HIP backend failed to load rather than the GPU being unavailable. `--list-devices` shows the loader error.
+
 ## Docker With MUSA
 
 Assuming one has the [mt-container-toolkit](https://developer.mthreads.com/musa/native) properly installed on Linux, `muBLAS` should be accessible inside the container.