[ROCm][AMD][Installation] Add mooncake build to rocm base image

[giuseppegrossi]

Purpose

Add AMD ROCm support for Mooncake to the vLLM ROCm build so the MooncakeConnector and MooncakeStoreConnector KV-transfer connectors are usable out of the box on ROCm images. Addresses issue #51193.

Related is the addition of LMCache in #51208 and follows the existing in-image "build the wheel from source" pattern already used for aiter, MoRI, and FlashAttention. It is a short-term solution: once Mooncake publishes an official ROCm wheel, this stage can be replaced.

What this does

Build follows Mooncake's own wheel-publishing recipe

The stage deliberately reuses Mooncake's upstream ROCm wheel-publishing path rather than inventing our own packaging:

• dependencies.sh — the project's own dependency bootstrapper.
• HIP_BUILD=1 ... scripts/build_wheel.sh — the exact script/flag Mooncake uses to emit a ROCm wheel (named mooncake_transfer_engine_rocm).

Tracking upstream's recipe means our wheel behaves like the one they will eventually publish, and the eventual migration to an official wheel is a straight swap.

Why the commit pin (not a tag)

ROCm wheel support was merged in kvcache-ai/Mooncake#3184 but has not been released. On every published tag, HIP_BUILD=1 is an unrecognized no-op and the build silently produces the CUDA-named mooncake_transfer_engine package instead. We therefore pin the merge commit df508641d76be20803349bf715d3bfc5205a7b58 (via git clone + git checkout, since --branch won't take a SHA) to get the ROCm package. Revisit and move to a tag once a release includes #3184.

Why we use the bundled dependencies.sh

Using Mooncake's own dependencies.sh instead of a hand-maintained apt install list means:

• We install the exact set and versions of build/runtime deps (yalantinglibs, etcd-cpp-apiv3, Go, etc.) that Mooncake's build expects.
• We don't drift from upstream and automatically pick up their dependency changes.
• It's the same path Mooncake's CI uses, so we inherit their validation instead of reverse-engineering it.

The one downside is that it installs Ubuntu 22.04's old patchelf, which we override immediately afterward (below).

Why we install a newer patchelf

dependencies.sh pulls in patchelf 0.14.3 (Ubuntu 22.04), which has a bug that corrupts RPATH entries when auditwheel repair rewrites them — it writes vendored library filenames into the RPATH instead of $ORIGIN-relative search paths. The failure is silent and dangerous:

• The wheel builds successfully and even imports inside the build stage, because cmake --install also scatters the libraries into /usr/local/lib where they're found by accident.
• The same wheel then fails at import time in a clean image with ImportError: lib...: cannot open shared object file.

Installing patchelf>=0.17 after dependencies.sh fixes the RPATH to the correct $ORIGIN-based form, matching Mooncake's published wheels. (Upstream's ROCm CI wouldn't catch this — their smoke test only runs mooncake_master --version and never imports the Python module.)

Other notes

• PLATFORM_TAG=auto: the base image's gcc-13 libstdc++ emits GLIBCXX_3.4.32, which the manylinux_2_35 policy the script would otherwise pick rejects. auto lets auditwheel select a compatible policy; the wheel ends up dual-tagged manylinux_2_35_x86_64.manylinux_2_39_x86_64 and installs cleanly on our glibc 2.35 image.
• Architecture-agnostic: the Mooncake transfer engine is host-side C++/RDMA with no gfx device kernels, so PYTORCH_ROCM_ARCH does not affect it. Exactly one portable wheel is produced regardless of the arch list.

Test plan

Build Image

docker build -f docker/Dockerfile.rocm --target vllm-openai \
  --build-arg BASE_IMAGE=rocm/vllm-dev:base \
  --build-arg ARG_PYTORCH_ROCM_ARCH=gfx950 \
  --build-arg max_jobs=192 \
  --network=host --progress plain \
  -t vllm-rocm-mooncake:pr .

Mooncake Cache

First run the image the was just created in a container.

docker run -it --rm --name mooncake-cache \
  --network=host --ipc=host --shm-size=16g \
  --device=/dev/kfd --device=/dev/dri --group-add video \
  --security-opt seccomp=unconfined \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --entrypoint bash vllm-rocm-mooncake:pr

Inside the container start the master, write a config file (matches docs' sample) and serve.

# 1. Start the Mooncake master
mooncake_master --port 50051 &

# 2. Embedded-mode store config
cat > /tmp/mooncake_config.json <<'EOF'
{
  "mode": "embedded",
  "metadata_server": "P2PHANDSHAKE",
  "master_server_address": "127.0.0.1:50051",
  "global_segment_size": "4GB",
  "local_buffer_size": "1GB",
  "protocol": "tcp",
  "device_name": "",
  "enable_offload": false
}
EOF

# 3. Launch vLLM with the MooncakeStoreConnector
MOONCAKE_CONFIG_PATH=/tmp/mooncake_config.json PYTHONHASHSEED=0 \
vllm serve Qwen/Qwen2.5-0.5B-Instruct \
  --port 8000 --max-model-len 4096 --gpu-memory-utilization 0.5 --enforce-eager \
  --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_both"}'

Once the serve is up, we can send a request.

curl -s http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" \
  -d '{"model":"Qwen/Qwen2.5-0.5B-Instruct","prompt":"The capital of France is","max_tokens":64}'

Test results

From within the container we get the following information.

=== versions.txt (mooncake) ===
MOONCAKE_BRANCH=df508641d76be20803349bf715d3bfc5205a7b58
MOONCAKE_REPO=https://github.com/kvcache-ai/Mooncake.git
=== mooncake wheel + engine ===
mooncake 0.3.12.post1 | SUPPORT_HIP True
=== store import ===
store import OK
=== master ===
mooncake_master version 0.3.12.post1 (git: df50864)
=== vllm + connectors ===
vllm 0.1.dev20069+g9610bea88
connectors: MooncakeStoreConnector + MooncakeConnector

And after running the prompt to the server we get the following sane result.

"text":" Paris, which has a population of 2.1 million people. The city was founded in the 13th century and has been an important center for French culture and history since its founding. It is located on the banks of the Seine River, surrounded by hills and forests.\n\nParis has many historical landmarks,"

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[AndreasKaratzas]

/amd-ci run nightly

[github-actions]

✅ Triggered Buildkite AMD CI #12535 for commit 340672677131.

[AndreasKaratzas]

/ci run

[github-actions]

✅ Triggered Buildkite CI #86737 for commit 340672677131.