chore(sglang): bump to 0.5.17 by jain-ria · Pull Request #13343 · ai-dynamo/dynamo

jain-ria · 2026-08-17T02:00:10Z

Summary

bump the Python SGLang diffusion extra from 0.5.16 to 0.5.17
bump the CUDA 13.0 runtime image to lmsysorg/sglang:v0.5.17-cu130-runtime
regenerate compatibility/release documentation and align the stock sidecar examples with 0.5.17
migrate post-resolution ServerArgs updates to SGLang's audited override() API, including GMS and snapshot/FPM paths
retain the decoded-tensor compatibility shim because the upstream issue remains in 0.5.17
start multimodal prefill before authorizing decode, propagate setup failures, track cancellation, and release transferred embeddings on every exit path

Upstream source validation used SGLang tag v0.5.17 at 29481685462732237d80d86076d6563e1f658102. The published CUDA runtime tags were verified as multi-architecture images: v0.5.17-cu130-runtime and v0.5.17-cu129-runtime. Dynamo main currently defines only the CUDA 13.0 SGLang runtime variant.

The supported CUDA engine window is 0.5.17/0.5.16; both expose ServerArgs.override() and _update_weight_version_if_provided(). The XPU image remains separately pinned to SGLang 0.5.11, so the compatibility helper keeps an explicitly documented legacy assignment fallback until that image reaches 0.5.16+.

Validation

exact clean environment: fresh Python 3.12 venv, editable SGLang source at tag v0.5.17, [diffusion] extra, rebuilt Dynamo native bindings
post-fix complete SGLang and GMS integration suites: 421 passed
all-files pre-commit: passed
release metadata generator/check: passed through pre-commit
every SGLang launch shell script: bash -n passed; changed deployment YAML parsed successfully
native Dynamo runtime wheel built and imported against exact SGLang 0.5.17

H100 validation with lmsysorg/sglang:v0.5.17-cu130-runtime:

PASS agg.sh
PASS agg_embed.sh
PASS agg_router.sh
PASS agg_vision.sh with inline image input
PASS disagg.sh
PASS disagg_router.sh
PASS disagg_same_gpu.sh
PASS diffusion_llada.sh with generated text
PASS image_diffusion.sh --model-path Tongyi-MAI/Z-Image-Turbo; generated a non-empty PNG through /v1/images/generations
PASS text-to-video-diffusion.sh with an MP4 output
PASS multimodal_epd.sh with inline image input
PASS multimodal_disagg.sh through the exact pre-merge case test_sglang_deployment[multimodal_disagg_qwen-4]; HTTP 200 and the response correctly described the pictured bus (repeated before and after the lifecycle fix)
PASS agg_multimodal_router.sh with inline image input
PASS agg_agent.sh with its default zai-org/GLM-4.7-Flash model (TP=2); model discovery and chat completion returned HTTP 200 with parsed reasoning/content fields

The earlier multimodal-disaggregated semantic-corruption observation was not reproducible in the clean exact-tag environment. Two independent temperature-zero runs returned the correct image description, including after the prefill lifecycle fix, so there is no remaining known GPU correctness issue from that report.

Summary by CodeRabbit

New Features
- Added compatibility with SGLang 0.5.17.
- Improved multimodal prefill processing, cancellation handling, and embedding cleanup.
- Improved weight-version updates and runtime configuration across supported deployments.
Bug Fixes
- Prevented failures during empty or interrupted multimodal streams.
- Improved reliability when engine submission or background processing fails.
Documentation
- Updated compatibility guidance, deployment examples, version references, and multimodal routing documentation to reflect SGLang 0.5.17.

Related Issues

Closes DEP (light): Updating the sglang dependency to v0.5.17 #13147

copy-pr-bot · 2026-08-17T02:00:13Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

jain-ria · 2026-08-17T18:00:06Z

/ok to test 6a32da6

github-actions · 2026-08-17T18:10:27Z

🌿 Fern Docs Preview: https://nvidia-preview-01a0216e-b64a-759d-807b-b7ab1a456be5.docs.buildwithfern.com/dynamo/dev

jain-ria · 2026-08-17T18:48:27Z

/ok to test 08c85e6

Signed-off-by: jain-ria <riajain@NVIDIA.com>

jain-ria · 2026-08-17T20:22:31Z

/ok to test 8c6f5b6

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

jthomson04

Two inline comments.

Signed-off-by: jain-ria <riajain@NVIDIA.com>

furionw

Thank you for working on this, @jain-ria .

Swipe4057 · 2026-08-20T05:20:34Z

Should I wait with the merge and update the dependency to 0.5.18 right away? The release branch is https://github.com/sgl-project/sglang/tree/release/v0.5.18

Signed-off-by: jain-ria <riajain@NVIDIA.com>

rmccorm4 · 2026-08-20T18:33:58Z

/ok to test 9e7d94e

ishandhanani · 2026-08-20T19:25:14Z

@Swipe4057 - gonna get this PR in first. And then will kick off another PR on top for the 0.5.18 branch. This PR has a lot of things to use new serverags.

BTW if you want to try it feel free

Signed-off-by: jain-ria <riajain@NVIDIA.com>

jain-ria · 2026-08-20T23:04:48Z

/ok to test c7cb8e4

jain-ria · 2026-08-21T19:27:06Z

@Swipe4057 started working on 0.5.18, see #13643

pull-request-size Bot added the size/M label Aug 17, 2026

github-actions Bot added build documentation Improvements or additions to documentation backend::sglang Relates to the sglang backend container labels Aug 17, 2026

jain-ria changed the title ~~build(sglang): bump runtime to 0.5.17~~ chore(sglang): bump to 0.5.17 Aug 17, 2026

github-actions Bot added chore and removed build labels Aug 17, 2026

jain-ria force-pushed the jain-ria/sgl-to-0.5.17-and-cleanups branch from 5909f89 to de4709c Compare August 17, 2026 16:21

pull-request-size Bot added size/L and removed size/M labels Aug 17, 2026

jain-ria force-pushed the jain-ria/sgl-to-0.5.17-and-cleanups branch from de4709c to 6a32da6 Compare August 17, 2026 17:59

github-actions Bot added the multimodal label Aug 17, 2026

marckarp mentioned this pull request Aug 17, 2026

feat: parse diffusion worker args natively from sglang's diffusion ServerArgs #13376

Closed

This comment has been minimized.

Sign in to view

jain-ria force-pushed the jain-ria/sgl-to-0.5.17-and-cleanups branch from 6a32da6 to 08c85e6 Compare August 17, 2026 18:48

jain-ria added 5 commits August 17, 2026 13:20

build(sglang): bump runtime to 0.5.17

3f3e126

Signed-off-by: jain-ria <riajain@NVIDIA.com>

fix(sglang): support read-only server args

f3777ac

Signed-off-by: jain-ria <riajain@NVIDIA.com>

build(sglang): align sidecar examples with 0.5.17

063048a

Signed-off-by: jain-ria <riajain@NVIDIA.com>

fix(sglang): preserve server arg updates across runtimes

cc2cf09

Signed-off-by: jain-ria <riajain@NVIDIA.com>

fix(sglang): order multimodal prefill bootstrap safely

8c6f5b6

Signed-off-by: jain-ria <riajain@NVIDIA.com>

jain-ria force-pushed the jain-ria/sgl-to-0.5.17-and-cleanups branch from 08c85e6 to 8c6f5b6 Compare August 17, 2026 20:22

jain-ria marked this pull request as ready for review August 17, 2026 21:25

jain-ria requested review from a team as code owners August 17, 2026 21:25

chore(recipes): refresh generated OpenAPI schema

2a0b096

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

ishandhanani requested a review from a team as a code owner August 18, 2026 17:22

furionw requested changes Aug 18, 2026

View reviewed changes

Comment thread components/src/dynamo/sglang/request_handlers/multimodal/worker_handler.py

Comment thread components/src/dynamo/sglang/request_handlers/multimodal/worker_handler.py

jthomson04 reviewed Aug 18, 2026

View reviewed changes

Comment thread components/src/dynamo/sglang/request_handlers/multimodal/worker_handler.py Outdated

Comment thread recipes/kustomize/components/dynamo-openapi/dynamo-openapi.json Outdated

nv-anants reviewed Aug 18, 2026

View reviewed changes

Comment thread container/context.yaml Outdated

galletas1712 reviewed Aug 18, 2026

View reviewed changes

Comment thread lib/gpu_memory_service/tests/test_sglang_patches.py Outdated

alec-flowers reviewed Aug 18, 2026

View reviewed changes

Comment thread docs/fern/pages/developer-guide/knowledge-base/modular-components/backends/sglang/multimodal.md

fix(sglang): arm multimodal cancellation before output

6534c35

Signed-off-by: jain-ria <riajain@NVIDIA.com>

pull-request-size Bot added size/XXL and removed size/XL labels Aug 19, 2026

jain-ria mentioned this pull request Aug 19, 2026

[BUG]: Multimodal embedding allocation leaks when item construction fails #13515

Open

furionw approved these changes Aug 20, 2026

View reviewed changes

pull-request-size Bot added size/XL and removed size/XXL labels Aug 20, 2026

jain-ria added 2 commits August 20, 2026 08:08

build(sglang): bump NIXL to 1.3.2

51bdae8

Signed-off-by: jain-ria <riajain@NVIDIA.com>

Merge branch 'main' into jain-ria/sgl-to-0.5.17-and-cleanups

9e7d94e

Signed-off-by: jain-ria <riajain@NVIDIA.com>

jain-ria force-pushed the jain-ria/sgl-to-0.5.17-and-cleanups branch from 8891b0e to 9e7d94e Compare August 20, 2026 15:12

nv-anants approved these changes Aug 20, 2026

View reviewed changes

galletas1712 approved these changes Aug 20, 2026

View reviewed changes

fix(ci): unblock SGLang bump checks

c7cb8e4

Signed-off-by: jain-ria <riajain@NVIDIA.com>

github-actions Bot added the actions label Aug 20, 2026

jain-ria merged commit 38d6116 into main Aug 21, 2026
121 checks passed

jain-ria deleted the jain-ria/sgl-to-0.5.17-and-cleanups branch August 21, 2026 00:05

jain-ria restored the jain-ria/sgl-to-0.5.17-and-cleanups branch August 21, 2026 17:44

marckarp mentioned this pull request Aug 22, 2026

feat: parse diffusion worker args natively from sglang's diffusion ServerArgs #13667

Draft

Uh oh!

Conversation

jain-ria commented Aug 17, 2026 • edited by nv-anants Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Summary by CodeRabbit

Related Issues

Uh oh!

copy-pr-bot Bot commented Aug 17, 2026

Uh oh!

jain-ria commented Aug 17, 2026

Uh oh!

github-actions Bot commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

jain-ria commented Aug 17, 2026

Uh oh!

jain-ria commented Aug 17, 2026

Uh oh!

Uh oh!

Uh oh!

jthomson04 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

furionw left a comment

Choose a reason for hiding this comment

Uh oh!

Swipe4057 commented Aug 20, 2026

Uh oh!

rmccorm4 commented Aug 20, 2026

Uh oh!

ishandhanani commented Aug 20, 2026

Uh oh!

jain-ria commented Aug 20, 2026

Uh oh!

Uh oh!

jain-ria commented Aug 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

jain-ria commented Aug 17, 2026 •

edited by nv-anants

Loading

github-actions Bot commented Aug 17, 2026 •

edited

Loading