Skip to content

chore(deps): bump sglang from 0.5.9/0.5.10rc0 to 0.5.10 #1064

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
CatherineSue merged 11 commits into from
Apr 9, 2026
Merged

chore(deps): bump sglang from 0.5.9/0.5.10rc0 to 0.5.10 #1064

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
aaa08a9
chore(deps): bump sglang from 0.5.9/0.5.10rc0 to 0.5.10
slin1237 Apr 8, 2026
78e2ccf
chore(ci): collapse sglang build matrix to v0.5.10 only
slin1237 Apr 8, 2026
958d756
fix(grpc): pass None for image_inputs in embedding requests
slin1237 Apr 8, 2026
fd71251
test(e2e): enable multimodal tests for sglang backend
CatherineSue Apr 8, 2026
6b1f1e5
fix(grpc): return MultimodalInputs dataclass instead of dict for mm_i…
CatherineSue Apr 8, 2026
573f7db
fix(grpc): update stale docstring for _parse_mm_inputs
CatherineSue Apr 8, 2026
e132750
chore(ci): add worker log collection for Go bindings E2E tests
CatherineSue Apr 8, 2026
819084c
fix(e2e): add --enable-multimodal for Llama-4-Scout on sglang 0.5.10
CatherineSue Apr 8, 2026
ae7db73
ci: avoid leaking go ffi loader env into sglang tests
CatherineSue Apr 8, 2026
2da33ad
Revert "ci: avoid leaking go ffi loader env into sglang tests"
CatherineSue Apr 8, 2026
94e09bc
fix(grpc): set correct default sampling params for Go gRPC client
key4ng Apr 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/release-sglang-docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Release SMG+SGLang Docker Image

run-name: >-
SMG+SGLang |
${{ github.event_name == 'push' && '3-version matrix' || format('base={0}', inputs.base_image_ref || 'lmsysorg/sglang:v0.5.9') }} |
${{ github.event_name == 'push' && 'base=lmsysorg/sglang:v0.5.10' || format('base={0}', inputs.base_image_ref || 'lmsysorg/sglang:v0.5.10') }} |
engine=${{ inputs.sglang_commit || 'latest' }} |
smg=${{ github.event_name == 'pull_request' && 'main' || inputs.smg_commit || 'v1.4.0' }} |
${{ github.event_name == 'pull_request' && 'dry-run |' || '' }}
Expand All @@ -23,8 +23,8 @@ on:
workflow_dispatch:
inputs:
base_image_ref:
description: 'Base image (e.g. lmsysorg/sglang:v0.5.9)'
default: 'lmsysorg/sglang:v0.5.9'
description: 'Base image (e.g. lmsysorg/sglang:v0.5.10)'
default: 'lmsysorg/sglang:v0.5.10'
required: true
type: string
sglang_repo:
Expand All @@ -47,7 +47,7 @@ on:
default: 'v1.4.0'
type: string
tag:
description: 'Override image tag (e.g. v1.4.0-sglang-v0.5.9)'
description: 'Override image tag (e.g. v1.4.0-sglang-v0.5.10)'
required: false
type: string

Expand All @@ -60,7 +60,7 @@ jobs:
strategy:
fail-fast: false
matrix:
base_image: ${{ github.event_name == 'push' && fromJSON('["lmsysorg/sglang:v0.5.9","lmsysorg/sglang:v0.5.8","lmsysorg/sglang:v0.5.7"]') || fromJSON(format('["{0}"]', inputs.base_image_ref || 'lmsysorg/sglang:v0.5.9')) }}
base_image: ${{ github.event_name == 'push' && fromJSON('["lmsysorg/sglang:v0.5.10"]') || fromJSON(format('["{0}"]', inputs.base_image_ref || 'lmsysorg/sglang:v0.5.10')) }}
uses: ./.github/workflows/_build-engine-image.yml
with:
engine: sglang
Expand Down
2 changes: 1 addition & 1 deletion docker/engine.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
# SMG_COMMIT - commit/ref for SMG_REPO ("latest" = HEAD)
#
# Usage:
# docker build --build-arg BASE_IMAGE_REF=lmsysorg/sglang:v0.5.9 \
# docker build --build-arg BASE_IMAGE_REF=lmsysorg/sglang:v0.5.10 \
# --build-arg ENGINE=sglang \
# --build-arg SMG_REPO=https://github.com/lightseekorg/smg \
# --build-arg SMG_COMMIT=v1.1.0 \
Expand Down Expand Up @@ -44,7 +44,7 @@
COPY scripts/installation/ /tmp/scripts/

# ── final stage: install SMG + conditionally install engine ──────────────────
FROM ${BASE_IMAGE_REF}

Check warning on line 47 in docker/engine.Dockerfile

View workflow job for this annotation

GitHub Actions / build (vllm/vllm-openai:v0.19.0) / build-and-push

Default value for global ARG results in an empty or invalid base image name 

InvalidDefaultArgInFrom: Default value for ARG ${BASE_IMAGE_REF} results in empty or invalid base image name
More info: https://docs.docker.com/go/dockerfile/rule/invalid-default-arg-in-from/

Check warning on line 47 in docker/engine.Dockerfile

View workflow job for this annotation

GitHub Actions / build (nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc8) / build-and-push

Default value for global ARG results in an empty or invalid base image name 

InvalidDefaultArgInFrom: Default value for ARG ${BASE_IMAGE_REF} results in empty or invalid base image name
More info: https://docs.docker.com/go/dockerfile/rule/invalid-default-arg-in-from/

Check warning on line 47 in docker/engine.Dockerfile

View workflow job for this annotation

GitHub Actions / build (lmsysorg/sglang:v0.5.10) / build-and-push

Default value for global ARG results in an empty or invalid base image name 

InvalidDefaultArgInFrom: Default value for ARG ${BASE_IMAGE_REF} results in empty or invalid base image name
More info: https://docs.docker.com/go/dockerfile/rule/invalid-default-arg-in-from/

ARG ENGINE=sglang
ARG BACKEND
Expand Down
4 changes: 2 additions & 2 deletions e2e_test/chat_completions/test_multimodal.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def _make_image_content(image_source: str) -> dict:
# =============================================================================


@pytest.mark.engine("vllm")
@pytest.mark.engine("vllm", "sglang")
@pytest.mark.gpu(1)
@pytest.mark.e2e
@pytest.mark.model("Qwen/Qwen3-VL-8B-Instruct")
Expand Down Expand Up @@ -170,7 +170,7 @@ def test_multi_images_mixed(self, model, setup_backend):
# =============================================================================


@pytest.mark.engine("vllm")
@pytest.mark.engine("vllm", "sglang")
@pytest.mark.gpu(4)
@pytest.mark.e2e
@pytest.mark.model("meta-llama/Llama-4-Scout-17B-16E-Instruct")
Expand Down
2 changes: 1 addition & 1 deletion grpc_servicer/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ classifiers = [

[project.optional-dependencies]
vllm = ["vllm>=0.19.0"]
sglang = ["sglang>=0.5.10rc0"]
sglang = ["sglang>=0.5.10"]

[project.urls]
Homepage = "https://github.com/lightseekorg/smg"
Expand Down
18 changes: 11 additions & 7 deletions grpc_servicer/smg_grpc_servicer/sglang/servicer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,11 @@
TokenizedEmbeddingReqInput,
TokenizedGenerateReqInput,
)
from sglang.srt.managers.schedule_batch import Modality, MultimodalDataItem
from sglang.srt.managers.schedule_batch import (
Modality,
MultimodalDataItem,
MultimodalInputs,
)
from sglang.srt.sampling.sampling_params import SamplingParams as SGLSamplingParams
from sglang.srt.server_args import ServerArgs
from sglang.utils import get_exception_traceback
Expand Down Expand Up @@ -351,7 +355,7 @@ async def HealthCheck(
rid=rid,
input_text="",
input_ids=[0],
image_inputs={"mm_items": []},
image_inputs=None,
token_type_ids=[0],
sampling_params=sampling_params,
)
Expand Down Expand Up @@ -768,8 +772,8 @@ def _decode_tensor_data(tensor_data):
arr = np.frombuffer(tensor_data.data, dtype=np_dtype).reshape(shape)
return torch.from_numpy(arr)

def _parse_mm_inputs(self, mm_proto) -> dict:
"""Parse proto MultimodalInputs into the mm_inputs dict expected by scheduler."""
def _parse_mm_inputs(self, mm_proto) -> MultimodalInputs:
"""Parse proto MultimodalInputs into the mm_inputs expected by scheduler."""
# Decode pixel_values from typed TensorData field
pixel_values = self._decode_tensor_data(mm_proto.pixel_values)

Expand All @@ -795,10 +799,10 @@ def _parse_mm_inputs(self, mm_proto) -> dict:
offsets=offsets,
)

result = {"mm_items": [mm_item]}
result = MultimodalInputs(mm_items=[mm_item])

if mm_proto.HasField("im_token_id"):
result["im_token_id"] = mm_proto.im_token_id
result.im_token_id = mm_proto.im_token_id

return result

Expand Down Expand Up @@ -827,7 +831,7 @@ def _convert_embed_request(
rid=grpc_req.request_id,
input_text=input_text,
input_ids=input_ids,
image_inputs={"mm_items": []},
image_inputs=None,
token_type_ids=list(grpc_req.token_type_ids),
sampling_params=sampling_params,
)
Expand Down
7 changes: 1 addition & 6 deletions scripts/ci_install_sglang.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,8 @@ else
fi

# Install SGLang with all dependencies
# TODO(changsu): Revert to `uv pip install "sglang[all]"` once sglang releases
# a stable version (>= 0.5.10) that includes the smg_grpc_servicer migration
# (sgl-project/sglang#20478). The rc0 is needed because the old built-in
# grpc_server.py references proto types (EmbedError, EmbedComplete) that were
# removed from our proto definitions.
echo "Installing SGLang..."
uv pip install --prerelease=allow "sglang[all]==0.5.10rc0"
uv pip install --prerelease=allow "sglang[all]==0.5.10"
Comment thread
slin1237 marked this conversation as resolved.
Resolved
Comment thread
CatherineSue marked this conversation as resolved.
Resolved
Comment thread
CatherineSue marked this conversation as resolved.
Resolved
Comment thread
CatherineSue marked this conversation as resolved.
Resolved
Comment thread
CatherineSue marked this conversation as resolved.
Resolved
Comment thread
CatherineSue marked this conversation as resolved.
Resolved

# Install flashinfer-jit-cache: sglang bundles flashinfer_python but only for attention ops.
# Multi-GPU models need trtllm_comm kernels (fused allreduce + layernorm) which FlashInfer
Expand Down
4 changes: 0 additions & 4 deletions scripts/installation/install-sglang.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,3 @@ set -e
SGL_SRC="${1:-/tmp/sglang-src}"
cd "${SGL_SRC}/python"
pip install --no-deps --force-reinstall --editable .

# Pin gRPC packages to 1.78.0 — grpcio 1.78.1 was yanked from PyPI
# TODO: remove when new SGLang 0.5.10 is released
pip install --force-reinstall grpcio==1.78.0 grpcio-reflection==1.78.0 grpcio-tools==1.78.0 grpcio-health-checking==1.78.0
Loading