Skip to content

chore(gb200): update dockerfile to handle fp4 disaggregation #8694

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 2, 2025
Merged

chore(gb200): update dockerfile to handle fp4 disaggregation #8694

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions docker/Dockerfile.gb200
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,10 @@ RUN mkdir -p /tmp/gdrcopy && cd /tmp \
RUN ln -sf /usr/lib/$(uname -m)-linux-gnu/libmlx5.so.1 /usr/lib/$(uname -m)-linux-gnu/libmlx5.so

# Clone and install SGLang
# NOTE: flashinfer v0.2.9rc1 is not installing for aarch64
WORKDIR /sgl-workspace
RUN python3 -m pip install --no-cache-dir --upgrade pip setuptools wheel html5lib six \
&& git clone https://github.com/sgl-project/sglang.git \
&& git clone --depth 1 https://github.com/sgl-project/sglang.git \
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Aug 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The git checkout to a specific commit has been removed. While using --depth 1 is a good optimization for reducing the clone size, removing the pinned commit means this Docker build will now pull the HEAD of the sglang repository's default branch. This can lead to non-reproducible builds and may introduce breaking changes unexpectedly.

For production stability and reproducible builds, it is highly recommended to pin dependencies to a specific git commit hash or tag.

&& cd sglang \
&& git checkout a167fd0bcb9ef4b0f4331a109e40c8cdc770b026 \
&& case "$CUDA_VERSION" in \
12.6.1) CUINDEX=126 ;; \
12.8.1) CUINDEX=128 ;; \
Expand Down Expand Up @@ -93,7 +91,7 @@ RUN wget https://developer.download.nvidia.com/compute/redist/nvshmem/3.3.9/sour
# Python tools
RUN python3 -m pip install --no-cache-dir \
datamodel_code_generator \
mooncake_transfer_engine==0.3.5 \
mooncake-transfer-engine==0.3.5 \
pre-commit \
pytest \
black \
Expand All @@ -103,6 +101,15 @@ RUN python3 -m pip install --no-cache-dir \
wheel \
scikit-build-core

# These will be automatically installed by future versions of flashinfer after 0.2.9rc2
RUN python3 -m pip install --no-cache-dir \
nvidia-cudnn-cu12 \
nvidia-cudnn-frontend

# Allows for FP4 disaggregation
RUN python3 -m pip install --no-cache-dir \
nixl

Comment on lines +104 to +112
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Aug 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To optimize the Docker image size and improve build performance, it's a best practice to combine consecutive RUN commands into a single layer. The two new pip install commands can be merged into one.

For further optimization, you could also merge these new packages into the preceding RUN pip install command on lines 92-102.

# These will be automatically installed by future versions of flashinfer after 0.2.9rc2
# Allows for FP4 disaggregation
RUN python3 -m pip install --no-cache-dir \
    nvidia-cudnn-cu12 \
    nvidia-cudnn-frontend \
    nixl

# Install development tools and utilities
RUN apt-get update && apt-get install -y \
gdb \
Expand Down
Loading