Add CI targets for ROCm 7.14 by superm1 · Pull Request #25775 · ggml-org/llama.cpp

superm1 · 2026-07-16T13:09:50Z

Overview

ROCm 7.14 is the first production release using TheRock build system. It can be installed using multi-arch deliverables from wheels, debs, rpms, tarballs or runfiles.

Add llama.cpp targets for both Linux and Windows to allow usage.

Additional information

Here are CI runs for the new Linux and Windows targets

Requirements

I have read and agree with the contributing guidelines
AI usage disclosure: NO

I manually created these changes based upon the previous targets.

CC @CISC

superm1 · 2026-07-16T16:17:02Z

CC @IMbackK as well from previous reviews.

IMbackK · 2026-07-17T08:40:17Z

yeah with 7.14 this is now the right thing to do. ill review this sunday

ggerganov · 2026-07-17T08:43:09Z

If I am reading this correctly, the Windows cache is ~4GB: https://github.com/superm1/llama.cpp/actions/runs/29467499221/job/87532475193#step:23:18

This is too big to be cached.

superm1 · 2026-07-17T13:18:48Z

If I am reading this correctly, the Windows cache is ~4GB: https://github.com/superm1/llama.cpp/actions/runs/29467499221/job/87532475193#step:23:18

This is too big to be cached.

Great catch.

I'm pretty sure I see the problem. When I copy-pasta'ed from the documentation I accidentally got the device libraries (which aren't need for build):

          python -m pip install --index-url https://repo.amd.com/rocm/whl-multi-arch/ "rocm[libraries,devel,device-all]"

I did not make that mistake on Ubuntu build.

          python -m pip install --index-url https://repo.amd.com/rocm/whl-multi-arch/ "rocm[libraries,devel]"

I'll adjust and this should sort it out.

superm1 · 2026-07-17T13:54:07Z

To avoid the cache on the 4GB binary on my fork I had to rename the key.

IMbackK

As we have problems with ci congestion we probubly dont want to build for both 7.14 and 7.2.

7.14 also includes support for gfx90c and gfx900 which we thus do want to build for.

superm1 · 2026-07-20T19:51:26Z

I've updated it, thanks. Here are the new job outputs.

https://github.com/superm1/llama.cpp/actions/runs/29753928301

apparle · 2026-07-23T02:01:50Z

7.14 also includes support for gfx1103 and it should also be added to gpu_targets.

IMbackK · 2026-07-23T15:29:38Z

afaik even though they build the gfx9 targets for windows, none of those can possibly work, as the kernel driver dosent support them, so we should also avoid building those objects.

superm1 · 2026-07-23T15:29:40Z

7.14 also includes support for gfx1103 and it should also be added to gpu_targets.

I've added this. I was also missing 1153.

FWIW In a similar PR to another ggml project it was noted that we should be using a pin with pip so that this doesn't move with the next ROCm release.

Now I'm using:

          python -m pip install --index-url https://repo.amd.com/rocm/whl-multi-arch/ "rocm[libraries,devel]==${ROCM_VERSION}"

superm1 · 2026-07-23T15:30:04Z

afaik even though they build the gfx9 targets for windows, none of those can possibly work, as the kernel driver dosent support them, so we should also avoid building those objects.

OK, I'll adjust.

superm1 · 2026-07-23T21:28:57Z

Here are the jobs with those changes for review:

https://github.com/superm1/llama.cpp/actions/runs/30030488460/job/89285913179
https://github.com/superm1/llama.cpp/actions/runs/30030488460/job/89285775126

IMbackK · 2026-07-24T07:19:42Z

Great ill take a look at the binaries the weekend

IMbackK

Binaries look good. Everyhing is fine from the rocm side
@CISC could you take a look from ci side.

metala · 2026-07-30T23:15:23Z

I've noticed performance downgrade on gfx1201 when switching from 7.2.4 to 7.14.

Edit. using a custom/updated rocm.Dockerfile

llama.cpp: (tag: b10184)

./llama-bench -ngl 99 -sm layer -fa on -m /models/gguf/Qwen3.6-27B-Q8_0.gguf -p 1024 -p 2048 -p 4096 -p 8192 -p 16384 -n 512 -r 3
ggml_cuda_init: found 2 ROCm devices (Total VRAM: 65248 MiB):
  Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32, VRAM: 32624 MiB
  Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32, VRAM: 32624 MiB

Kernel: Linux 7.1.3+deb14-amd64
GPUs are running on PCIe4.0 x8/x8 bifurcation on Zen3 CPU.

ROCm 7.2.4

model	size	params	backend	ngl	fa	test	t/s
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	1	pp1024	1620.71 ± 30.09
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	1	pp2048	1897.21 ± 2.55
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	1	pp4096	2001.76 ± 2.77
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	1	pp8192	1952.42 ± 3.76
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	1	pp16384	1757.94 ± 2.28
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	1	tg512	19.22 ± 0.01

ROCm 7.14

model	size	params	backend	ngl	fa	test	t/s
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	1	pp1024	1618.55 ± 1.08
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	1	pp2048	1870.10 ± 2.33
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	1	pp4096	1969.74 ± 3.11
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	1	pp8192	1913.75 ± 4.50
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	1	pp16384	1719.52 ± 1.81
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	1	tg512	19.18 ± 0.03

split-mode: none (single-GPU)
ROCm 7.2.4

model	size	params	backend	ngl	sm	fa	test	t/s
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	none	1	pp1024	1100.62 ± 1.00
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	none	1	pp2048	1077.46 ± 0.75
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	none	1	pp4096	1032.18 ± 1.92
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	none	1	pp8192	954.66 ± 1.20
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	none	1	pp16384	838.50 ± 0.26
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	none	1	tg512	19.38 ± 0.02

ROCm 7.14

model	size	params	backend	ngl	sm	fa	test	t/s
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	none	1	pp1024	1096.49 ± 0.77
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	none	1	pp2048	1072.14 ± 1.71
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	none	1	pp4096	1025.32 ± 2.34
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	none	1	pp8192	944.36 ± 2.18
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	none	1	pp16384	822.88 ± 1.11
qwen35 27B Q8_0	26.62 GiB	26.90 B	ROCm	99	none	1	tg512	19.36 ± 0.01

superm1 · 2026-07-31T00:02:03Z

I've noticed performance downgrade on gfx1201 when switching from 7.2.4 to 7.14.

Edit. using a custom/updated rocm.Dockerfile

Can you please qualify that? Was it the exact same llama.cpp build?

How is it measured and how much of a drop?

metala · 2026-07-31T00:06:13Z

Can you please qualify that? Was it the exact same llama.cpp build?

b10184

How is it measured and how much of a drop?

llama-bench and llama-benchy. I've added llama-bench results above.

My benchmarks show consistently 1% to 3% drop in tokens/s in prompt processing. The token generation is not affected. I've also tested Ubuntu 24.04 (the base image used in 7.2.4) with ROCm 7.14 and it got even worse.

I will try to publish the Dockerfile.
7.2.4: https://pgit.metala.org/llama.cpp.git/diff/.devops/rocm.Dockerfile?h=rocm
7.14: https://pgit.metala.org/llama.cpp.git/diff/.devops/rocm.Dockerfile?h=rocm7.14

ROCm 7.14 is the first production release using TheRock build system. It can be installed using multi-arch deliverables from wheels, debs, rpms, tarballs or runfiles. Adjust ROCm targets for Linux and Windows to use this instead.

Move the shared windows-setup-rocm composite action from the HIP SDK PRO Edition installer to the multi-arch ROCm wheels (rocm[libraries,devel]). The wheel-install logic that previously lived inline in release.yml is now in the shared action, and both build-cache.yml and release.yml call it. Also migrate the build-cuda-windows.yml hip job to the same wheel-based layout (cache path/key, rocm-sdk environment setup, llvm/bin compiler paths) so it keeps working after the action's contract changed; drop its now-unused ROCm 7.2.1 rocWMMA download and stale include path.

CISC · 2026-08-10T14:52:04Z

@superm1 Can you cancel all the old release jobs so the last one can start?

superm1 · 2026-08-10T15:50:03Z

Yup; just did. Looks like the one I changed for Windows passed now too.

https://github.com/superm1/llama.cpp/actions/runs/31397302547/job/93503759627

ahoplock · 2026-08-11T18:49:32Z

Lost my ROCm0 device in any version since this released as b10356 (Windows 10).

$ llama-server --version
version: 10355 (dd1ea5243)
built with Clang 20.1.8 for Windows x86_64
$ llama-server --list-devices
Available devices:
  ROCm0: AMD Radeon RX 9070 XT (16304 MiB, 16142 MiB free)
  Vulkan0: AMD Radeon RX 9070 XT (16304 MiB, 15416 MiB free)

$ llama-server --version
version: 10356 (0666ad2b2)
built with Clang 20.1.8 for Windows x86_64
$ llama-server --list-devices
Available devices:
  Vulkan0: AMD Radeon RX 9070 XT (16304 MiB, 15416 MiB free)

superm1 · 2026-08-11T20:01:38Z

Lost my ROCm0 device in any version since this released as b10356 (Windows 10).

$ llama-server --version
version: 10355 (dd1ea5243)
built with Clang 20.1.8 for Windows x86_64
$ llama-server --list-devices
Available devices:
  ROCm0: AMD Radeon RX 9070 XT (16304 MiB, 16142 MiB free)
  Vulkan0: AMD Radeon RX 9070 XT (16304 MiB, 15416 MiB free)

$ llama-server --version
version: 10356 (0666ad2b2)
built with Clang 20.1.8 for Windows x86_64
$ llama-server --list-devices
Available devices:
  Vulkan0: AMD Radeon RX 9070 XT (16304 MiB, 15416 MiB free)

What version of ROCm are you using? Is it in PATH?

ahoplock · 2026-08-11T20:38:57Z

What version of ROCm are you using? Is it in PATH?

Never mind...AMD's 7.14 install guide bricked my system path.

ahoplock · 2026-08-11T20:47:58Z

I guess never mind my never mind as it now finds the device but fails to get memory.

$ llama-server --list-devices
Available devices:
  ROCm0: AMD Radeon RX 9070 XT (0 MiB, 0 MiB free)
0.00.153.916 W ggml_backend_cuda_device_get_memory: cudaMemGetInfo failed (invalid argument), returning 0/0
  Vulkan0: AMD Radeon RX 9070 XT (16304 MiB, 15416 MiB free)

MrDrMcCoy · 2026-08-11T21:15:38Z

I've been having an issue for a while now where newer ROCm releases refuse to touch VRAM and insist on slowly streaming from system RAM. Only thing left for me to really try is fully wiping my system and starting fresh, which I would prefer to not have to do.

superm1 · 2026-08-11T21:19:52Z

I guess never mind my never mind as it now finds the device but fails to get memory.

$ llama-server --list-devices
Available devices:
  ROCm0: AMD Radeon RX 9070 XT (0 MiB, 0 MiB free)
0.00.153.916 W ggml_backend_cuda_device_get_memory: cudaMemGetInfo failed (invalid argument), returning 0/0
  Vulkan0: AMD Radeon RX 9070 XT (16304 MiB, 15416 MiB free)

This sounds like the behavior of having a system-wide hip dll

ahoplock · 2026-08-11T22:51:40Z

This sounds like the behavior of having a system-wide hip dll

Thanks for pointing me in the right direction. Looks like C:\Windows\System32\amdhip64_7.dll (guessing from the AMD Adrenalin driver based off of file creation dates) was getting loaded, so I just copied the one from C:\TheRock\build\bin over to where my llama-server lives and it loads models as before.

I have all of the HIP related env vars/paths set from AMD's ROCm 7.14 install guide, so I'm not sure what's wrong.

* Switch ROCm from 7.2.1 to 7.14 ROCm 7.14 is the first production release using TheRock build system. It can be installed using multi-arch deliverables from wheels, debs, rpms, tarballs or runfiles. Adjust ROCm targets for Linux and Windows to use this instead. * ci: switch all other Windows ROCm jobs to ROCm 7.14 wheels Move the shared windows-setup-rocm composite action from the HIP SDK PRO Edition installer to the multi-arch ROCm wheels (rocm[libraries,devel]). The wheel-install logic that previously lived inline in release.yml is now in the shared action, and both build-cache.yml and release.yml call it. Also migrate the build-cuda-windows.yml hip job to the same wheel-based layout (cache path/key, rocm-sdk environment setup, llvm/bin compiler paths) so it keeps working after the action's contract changed; drop its now-unused ROCm 7.2.1 rocWMMA download and stale include path.

superm1 requested a review from a team as a code owner July 16, 2026 13:09

github-actions Bot added the devops improvements to build systems and github actions label Jul 16, 2026

IMbackK self-assigned this Jul 17, 2026

superm1 force-pushed the master branch 2 times, most recently from 9122605 to 61cbc9c Compare July 17, 2026 13:49

IMbackK requested changes Jul 20, 2026

View reviewed changes

superm1 force-pushed the master branch 2 times, most recently from 7eced8f to fc237f0 Compare July 20, 2026 15:09

superm1 force-pushed the master branch 2 times, most recently from 53b2c54 to 23eafcd Compare July 23, 2026 15:28

superm1 force-pushed the master branch 4 times, most recently from 380cf93 to 8df3f3d Compare July 23, 2026 17:41

IMbackK approved these changes Jul 26, 2026

View reviewed changes

superm1 mentioned this pull request Jul 29, 2026

HIP: Add gfx1152 to ubuntu-22-rocm matrix as and windows build matrix… #26256

Closed

CISC reviewed Aug 3, 2026

View reviewed changes

Comment thread .github/workflows/release.yml Outdated

Comment thread .github/workflows/release.yml Outdated

Comment thread .github/workflows/release.yml Outdated

Comment thread .github/workflows/release.yml

superm1 force-pushed the master branch 2 times, most recently from 5db6a19 to 78ae4a9 Compare August 10, 2026 00:49

CISC reviewed Aug 10, 2026

View reviewed changes

Comment thread .github/workflows/release.yml Outdated

Switch ROCm from 7.2.1 to 7.14

2414237

ROCm 7.14 is the first production release using TheRock build system. It can be installed using multi-arch deliverables from wheels, debs, rpms, tarballs or runfiles. Adjust ROCm targets for Linux and Windows to use this instead.

superm1 force-pushed the master branch from 78ae4a9 to 2414237 Compare August 10, 2026 12:00

CISC approved these changes Aug 10, 2026

View reviewed changes

CISC reviewed Aug 10, 2026

View reviewed changes

Comment thread .github/workflows/release.yml

CISC approved these changes Aug 10, 2026

View reviewed changes

CISC merged commit 0666ad2 into ggml-org:master Aug 10, 2026
3 checks passed

CISC mentioned this pull request Aug 11, 2026

ci : add windows-rocm to check-release #26897

Merged

satoyon mentioned this pull request Aug 12, 2026

Misc. bug: Pre-built Windows ROCm binary does not detect GPU #26929

Closed

ahoplock mentioned this pull request Aug 12, 2026

Misc. bug: ROCm-7.14 - > 'error while loading shared libraries: libhipblas.so.3' #25807

Open

DanoPTT mentioned this pull request Aug 13, 2026

win-rocm-7.14 Windows release missing hipblas.dll — GPU not detected, --list-devices returns empty #26996

Open

jimw567 mentioned this pull request Aug 18, 2026

ci: re-enable ubuntu-22-rocm release build with larger ccache AMD-Ecosystem/llama.cpp#91

Open

3 tasks

Uh oh!

Conversation

superm1 commented Jul 16, 2026

Overview

Additional information

Requirements

Uh oh!

superm1 commented Jul 16, 2026

Uh oh!

IMbackK commented Jul 17, 2026

Uh oh!

ggerganov commented Jul 17, 2026

Uh oh!

superm1 commented Jul 17, 2026

Uh oh!

superm1 commented Jul 17, 2026

Uh oh!

IMbackK left a comment

Choose a reason for hiding this comment

Uh oh!

superm1 commented Jul 20, 2026

Uh oh!

apparle commented Jul 23, 2026

Uh oh!

IMbackK commented Jul 23, 2026

Uh oh!

superm1 commented Jul 23, 2026

Uh oh!

superm1 commented Jul 23, 2026

Uh oh!

superm1 commented Jul 23, 2026

Uh oh!

IMbackK commented Jul 24, 2026

Uh oh!

IMbackK left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

metala commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

superm1 commented Jul 31, 2026

Uh oh!

metala commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CISC commented Aug 10, 2026

Uh oh!

superm1 commented Aug 10, 2026

Uh oh!

Uh oh!

ahoplock commented Aug 11, 2026

Uh oh!

superm1 commented Aug 11, 2026

Uh oh!

ahoplock commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ahoplock commented Aug 11, 2026

Uh oh!

MrDrMcCoy commented Aug 11, 2026

Uh oh!

superm1 commented Aug 11, 2026

Uh oh!

ahoplock commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

IMbackK left a comment •

edited

Loading

metala commented Jul 30, 2026 •

edited

Loading

metala commented Jul 31, 2026 •

edited

Loading

ahoplock commented Aug 11, 2026 •

edited

Loading