Skip to content

[v0.5.10][1] Bump sglang to v0.5.10 #898

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yueming-yuan merged 14 commits into from
Apr 9, 2026
Merged

[v0.5.10][1] Bump sglang to v0.5.10 #898

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
38458fc
Bump sglang to v0.5.10
yueming-yuan Apr 6, 2026
fee8ee2
Fix PyJWT uninstall conflict with v0.5.10 base image
yueming-yuan Apr 6, 2026
dedb1fa
Fix PyJWT: rm debian package files before pip install
yueming-yuan Apr 6, 2026
6d0a8e3
Retrigger CI
yueming-yuan Apr 6, 2026
ffd110b
Fix FastAPI add_event_handler removed in newer versions
yueming-yuan Apr 6, 2026
3cf7a9f
Fix OpenAIServingChat missing reasoning_parser in test mock
yueming-yuan Apr 6, 2026
44e83de
Fix mask_utils for transformers >=5.0 BatchEncoding return type
yueming-yuan Apr 6, 2026
1a2d5e8
Format
yueming-yuan Apr 6, 2026
14a0cdb
Merge branch 'fix/mask-utils-transformers-v5' into bump-sglang-v0.5.10
yueming-yuan Apr 6, 2026
d549b26
Revert "Merge branch 'fix/mask-utils-transformers-v5' into bump-sglan…
yueming-yuan Apr 6, 2026
fbec694
Use Megatron-Bridge fix/rope-theta branch for transformers 5.x compat
yueming-yuan Apr 7, 2026
1b1e83f
Disable flashinfer allreduce fusion for H100 colocate to save NVLS ha…
yueming-yuan Apr 7, 2026
e848032
Revert "Disable flashinfer allreduce fusion for H100 colocate to save…
yueming-yuan Apr 7, 2026
710439f
Revert CI test docker/branch and Dockerfile changes
yueming-yuan Apr 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 24 additions & 24 deletions .github/workflows/pr-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ jobs:
if: (github.event_name == 'workflow_dispatch') || (github.event.pull_request)
runs-on: self-hosted
container:
image: radixark/miles:dev
image: radixark/miles:test-v0.5.10
options: >
--gpus all
--ipc=host
Expand Down Expand Up @@ -132,7 +132,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down Expand Up @@ -170,7 +170,7 @@ jobs:
if: (github.event_name == 'workflow_dispatch') || (github.event.pull_request && contains(github.event.pull_request.labels.*.name, 'run-unit-test'))
runs-on: self-hosted
container:
image: radixark/miles:dev
image: radixark/miles:test-v0.5.10
options: >
--gpus all
--ipc=host
Expand Down Expand Up @@ -244,7 +244,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down Expand Up @@ -282,7 +282,7 @@ jobs:
if: (github.event_name == 'workflow_dispatch') || (github.event.pull_request && contains(github.event.pull_request.labels.*.name, 'run-ci-sglang'))
runs-on: self-hosted
container:
image: radixark/miles:dev
image: radixark/miles:test-v0.5.10
options: >
--gpus all
--ipc=host
Expand Down Expand Up @@ -356,7 +356,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down Expand Up @@ -394,7 +394,7 @@ jobs:
if: (github.event_name == 'workflow_dispatch') || (github.event.pull_request && contains(github.event.pull_request.labels.*.name, 'run-ci-short'))
runs-on: self-hosted
container:
image: radixark/miles:dev
image: radixark/miles:test-v0.5.10
options: >
--gpus all
--ipc=host
Expand Down Expand Up @@ -468,7 +468,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down Expand Up @@ -506,7 +506,7 @@ jobs:
if: (github.event_name == 'workflow_dispatch') || (github.event.pull_request && contains(github.event.pull_request.labels.*.name, 'run-ci-fsdp'))
runs-on: self-hosted
container:
image: radixark/miles:dev
image: radixark/miles:test-v0.5.10
options: >
--gpus all
--ipc=host
Expand Down Expand Up @@ -580,7 +580,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down Expand Up @@ -618,7 +618,7 @@ jobs:
if: (github.event_name == 'workflow_dispatch') || (github.event.pull_request && contains(github.event.pull_request.labels.*.name, 'run-ci-megatron'))
runs-on: self-hosted
container:
image: radixark/miles:dev
image: radixark/miles:test-v0.5.10
options: >
--gpus all
--ipc=host
Expand Down Expand Up @@ -692,7 +692,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down Expand Up @@ -730,7 +730,7 @@ jobs:
if: (github.event_name == 'workflow_dispatch') || (github.event.pull_request && contains(github.event.pull_request.labels.*.name, 'run-ci-precision'))
runs-on: self-hosted
container:
image: radixark/miles:dev
image: radixark/miles:test-v0.5.10
options: >
--gpus all
--ipc=host
Expand Down Expand Up @@ -804,7 +804,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down Expand Up @@ -842,7 +842,7 @@ jobs:
if: (github.event_name == 'workflow_dispatch') || (github.event.pull_request && contains(github.event.pull_request.labels.*.name, 'run-ci-ckpt'))
runs-on: self-hosted
container:
image: radixark/miles:dev
image: radixark/miles:test-v0.5.10
options: >
--gpus all
--ipc=host
Expand Down Expand Up @@ -916,7 +916,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down Expand Up @@ -954,7 +954,7 @@ jobs:
if: (github.event_name == 'workflow_dispatch') || (github.event.pull_request && contains(github.event.pull_request.labels.*.name, 'run-ci-long'))
runs-on: self-hosted
container:
image: radixark/miles:dev
image: radixark/miles:test-v0.5.10
options: >
--gpus all
--ipc=host
Expand Down Expand Up @@ -1028,7 +1028,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down Expand Up @@ -1066,7 +1066,7 @@ jobs:
if: (github.event_name == 'workflow_dispatch') || (github.event.pull_request && contains(github.event.pull_request.labels.*.name, 'run-ci-lora'))
runs-on: self-hosted
container:
image: radixark/miles:dev
image: radixark/miles:test-v0.5.10
options: >
--gpus all
--ipc=host
Expand Down Expand Up @@ -1140,7 +1140,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down Expand Up @@ -1178,7 +1178,7 @@ jobs:
if: (github.event_name == 'workflow_dispatch') || (github.event.pull_request && contains(github.event.pull_request.labels.*.name, 'run-ci-low-precision'))
runs-on: self-hosted
container:
image: radixark/miles:dev
image: radixark/miles:test-v0.5.10
options: >
--gpus all
--ipc=host
Expand Down Expand Up @@ -1252,7 +1252,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down Expand Up @@ -1357,7 +1357,7 @@ jobs:
if: (github.event_name == 'workflow_dispatch') || (github.event.pull_request && contains(github.event.pull_request.labels.*.name, 'run-ci-image'))
runs-on: self-hosted
container:
image: radixark/miles:dev
image: radixark/miles:test-v0.5.10
options: >
--gpus all
--ipc=host
Expand Down Expand Up @@ -1431,7 +1431,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/pr-test.yml.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<% set default_image = 'radixark/miles:dev' %>
<% set default_image = 'radixark/miles:test-v0.5.10' %>

<% set fsdp_tests = [
{'test_file': 'e2e/fsdp/test_qwen3_4B_fsdp_true_on_policy.py', 'num_gpus': 8},
Expand Down Expand Up @@ -237,7 +237,7 @@ jobs:

# Defaults
[ -z "$MEGATRON_PR" ] && MEGATRON_PR="miles-main"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles"
[ -z "$SGLANG_PR" ] && SGLANG_PR="sglang-miles-v0.5.10"

# Convert "#N" PR syntax to git fetch ref: "pull/N/head"
resolve_fetch_ref() {
Expand Down
10 changes: 5 additions & 5 deletions docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@
#
# 2. radixark/miles:dev-cu13-arm64
# build-arg:ENABLE_CUDA_13=1 \
# build-arg:SGLANG_IMAGE_TAG=v0.5.9-cu130-arm64 \
# build-arg:SGLANG_IMAGE_TAG=v0.5.10-cu130 \

@gemini-code-assist gemini-code-assist Bot Apr 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The SGLANG_IMAGE_TAG in the example comment for the arm64 build has lost its -arm64 suffix. Since the header (line 4) and the WHEELS_TAG (line 7) both specify arm64/aarch64, the image tag should likely include the architecture suffix to ensure the correct base image is pulled, consistent with the previous version.

#    build-arg:SGLANG_IMAGE_TAG=v0.5.10-cu130-arm64 \

# build-arg:WHEELS_TAG=cu130-aarch64 \

ARG SGLANG_IMAGE_TAG=v0.5.9
ARG SGLANG_IMAGE_TAG=v0.5.10
FROM lmsysorg/sglang:${SGLANG_IMAGE_TAG} AS sglang

# ======================================== Arguments =============================================

ARG SGLANG_BRANCH=sglang-miles
ARG SGLANG_BRANCH=sglang-miles-v0.5.10
ARG SGLANG_COMMIT=""

ARG MEGATRON_REPO=radixark/Megatron-LM
Expand Down Expand Up @@ -83,12 +83,12 @@ RUN git clone https://github.com/${MEGATRON_REPO}.git --recursive -b ${MEGATRON_
RUN pip install git+https://github.com/fzyzcjy/torch_memory_saver.git@d64a639 --no-cache-dir --force-reinstall
# RUN pip install git+https://github.com/fzyzcjy/Megatron-Bridge.git@dev_rl --no-build-isolation
RUN pip install "nvidia-modelopt[torch]>=0.37.0" --no-build-isolation
RUN pip install git+https://github.com/yushengsu-thu/Megatron-Bridge.git@merged-megatron-0.16.0rc0-miles --no-deps --no-build-isolation
RUN pip install git+https://github.com/yushengsu-thu/Megatron-Bridge.git@fix/rope-theta-transformers-v5 --no-deps --no-build-isolation
RUN pip install megatron-energon --no-deps
RUN pip install multi-storage-client --no-deps

COPY requirements.txt /tmp/requirements.txt
RUN pip install -r /tmp/requirements.txt
RUN rm -rf /usr/lib/python3/dist-packages/jwt /usr/lib/python3/dist-packages/PyJWT* && pip install -r /tmp/requirements.txt

# https://github.com/pytorch/pytorch/issues/168167
RUN if [ "${ENABLE_CUDA_13}" = "1" ]; then \
Expand Down
2 changes: 1 addition & 1 deletion miles/rollout/session/session_server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ def __init__(self, args, backend_url: str):
)

# Close the httpx connection pool when uvicorn shuts down to avoid FD leaks.
self.app.add_event_handler("shutdown", self.client.aclose)
self.app.router.on_shutdown.append(self.client.aclose)

setup_session_routes(self.app, self, args)

Expand Down
2 changes: 1 addition & 1 deletion miles/router/router.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ def __init__(self, args, verbose=False):
self.verbose = verbose

self.app = FastAPI()
self.app.add_event_handler("startup", self._start_background_health_check)
self.app.router.on_startup.append(self._start_background_health_check)

# URL -> Active Request Count (load state)
self.worker_request_counts: dict[str, int] = {}
Expand Down
1 change: 1 addition & 0 deletions tests/fast/utils/chat_template_utils/test_template.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ def _make_serving(tokenizer) -> OpenAIServingChat:
serving.use_dpsk_v32_encoding = False
serving.is_gpt_oss = False
serving.tool_call_parser = None
serving.reasoning_parser = None
return serving


Expand Down
Loading