Skip to content

fix(examples): update Triton example for CUDA 13 + fix libdcgm copy (DYN-3697) #12577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dagil-nvidia merged 2 commits into from
Aug 3, 2026
Merged

fix(examples): update Triton example for CUDA 13 + fix libdcgm copy (DYN-3697) #12577

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1e0c758
fix(examples): update Triton example to CUDA 13-compatible image (DYN…
nealvaidya Jul 31, 2026
9208022
fix(examples): fix libdcgm copy in Triton Dockerfile for 25.10 (DYN-3…
nealvaidya Aug 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions examples/backends/tritonserver/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,19 @@
# SPDX-License-Identifier: Apache-2.0

ARG DYNAMO_BASE_IMAGE="dynamo-base:latest"
ARG TRITON_SERVER_IMAGE="nvcr.io/nvidia/tritonserver:25.01-py3"
ARG TRITON_SERVER_IMAGE="nvcr.io/nvidia/tritonserver:25.10-py3"

FROM ${TRITON_SERVER_IMAGE} AS triton_source
# Ensure DCGM directory exists so COPY never fails when the source image omits it
RUN mkdir -p /usr/local/dcgm

FROM ${DYNAMO_BASE_IMAGE} AS dynamo_base

COPY --from=triton_source /opt/tritonserver /opt/tritonserver
COPY --from=triton_source /usr/local/dcgm /usr/local/dcgm
COPY --from=triton_source /lib/x86_64-linux-gnu/libdcgm*.so* /lib/x86_64-linux-gnu/
USER root
RUN --mount=type=bind,from=triton_source,source=/usr/lib/x86_64-linux-gnu,target=/triton_usr_lib \
find /triton_usr_lib -name "libdcgm*.so*" -exec cp -a {} /lib/x86_64-linux-gnu/ \;
Comment thread
nealvaidya marked this conversation as resolved.
ENV BACKEND_DIR=/opt/tritonserver/backends
ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64:/opt/tritonserver/lib:/opt/tritonserver/backends:/usr/local/dcgm/lib64
ENV PATH=/opt/tritonserver/bin:$PATH
Expand All @@ -24,3 +28,5 @@ USER dynamo

RUN uv pip install --no-cache-dir tritonclient[grpc]
RUN uv pip install /opt/tritonserver/python/triton*.whl
# Validate CUDA ABI compatibility at build time
RUN python3 -c "import tritonserver"
Comment thread
nealvaidya marked this conversation as resolved.
15 changes: 10 additions & 5 deletions examples/backends/tritonserver/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,21 +38,26 @@ This example shows how to run Triton Server models through Dynamo's distributed
From the Dynamo repository root:

```bash
# Build the base Dynamo image
# Build the base Dynamo image (CUDA 13.0)
python container/render.py --framework=dynamo --target=runtime --output-short-filename
docker build -f container/rendered.Dockerfile -t dynamo-base:latest .

# Build the Triton worker image
cd examples/backends/tritonserver
docker build -t dynamo-triton:latest .
# Build the Triton worker image (defaults to tritonserver:25.10-py3, CUDA 13-compatible)
docker build \
--build-arg DYNAMO_BASE_IMAGE=dynamo-base:latest \
-t dynamo-triton:latest \
examples/backends/tritonserver/
```

> [!NOTE]
> The default `TRITON_SERVER_IMAGE` is `nvcr.io/nvidia/tritonserver:25.10-py3`

#### Step 2: Run the Container

```bash
docker run --rm -it --gpus all --network host \
dynamo-triton:latest \
./examples/backends/tritonserver/launch/identity.sh
/workspace/launch/identity.sh
Comment thread
nealvaidya marked this conversation as resolved.
```

#### Step 3: Test the Deployment
Expand Down
Loading