feat: add native gRPC sidecar module launcher - #31076
Merged
Merged
Conversation
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
ishandhanani
marked this pull request as ready for review
July 13, 2026 22:46
ishandhanani
requested review from
CatherineSue,
JustinTong0323,
ispobock,
merrymercy and
slin1237
as code owners
July 13, 2026 22:46
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
ishandhanani
force-pushed
the
codex/enable-dynamo-official
branch
from
July 13, 2026 22:52
6eb853b to
197b70a
Compare
ishandhanani
marked this pull request as draft
July 13, 2026 22:52
ishandhanani
marked this pull request as ready for review
July 14, 2026 06:54
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Collaborator
Author
|
/tag-and-rerun-ci |
ishandhanani
commented
Jul 14, 2026
Kangyan-Zhou
approved these changes
Jul 16, 2026
ishandhanani
commented
Jul 19, 2026
ishandhanani
left a comment
Collaborator
Author
There was a problem hiding this comment.
@connorcarpenter15 PTAL and review this PR
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
ishandhanani
force-pushed
the
codex/enable-dynamo-official
branch
from
July 22, 2026 10:18
4277853 to
9ac19ba
Compare
Collaborator
Author
|
Relevant CI has passed. Getting this in as it does not affect any GPU pieces at all |
Qiaolin-Yu
pushed a commit
that referenced
this pull request
Jul 22, 2026
Zhylkaaa
pushed a commit
to Zhylkaaa/sglang
that referenced
this pull request
Jul 29, 2026
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com> Signed-off-by: Connor Carpenter <connorc@nvidia.com> Co-authored-by: Connor Carpenter <connorc@nvidia.com>
jinzhenfan
pushed a commit
to jinzhenfan/sglang
that referenced
this pull request
Jul 29, 2026
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com> Signed-off-by: Connor Carpenter <connorc@nvidia.com> Co-authored-by: Connor Carpenter <connorc@nvidia.com>
Chronostasys
pushed a commit
to MindLab-Research/sglang
that referenced
this pull request
Aug 24, 2026
…ect#31076) (sgl-project#32074) Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com> Signed-off-by: Connor Carpenter <connorc@nvidia.com> Co-authored-by: Connor Carpenter <connorc@nvidia.com>
OrangeRedeng
pushed a commit
to OrangeRedeng/sglang
that referenced
this pull request
Aug 24, 2026
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com> Signed-off-by: Connor Carpenter <connorc@nvidia.com> Co-authored-by: Connor Carpenter <connorc@nvidia.com>
jakki-amd
pushed a commit
to jakki-amd/sglang
that referenced
this pull request
Sep 9, 2026
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com> Signed-off-by: Connor Carpenter <connorc@nvidia.com> Co-authored-by: Connor Carpenter <connorc@nvidia.com>
Atituiset
pushed a commit
to Atituiset/sglang
that referenced
this pull request
Sep 10, 2026
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com> Signed-off-by: Connor Carpenter <connorc@nvidia.com> Co-authored-by: Connor Carpenter <connorc@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an optional
--sidecar <python.module>to launch a local companion process after SGLang's native gRPC listener starts. This removes the Dynamo-specific server contract while preserving SGLang-owned child supervision and lets provider packages remain optional.How This Was Implemented
main(argv)receives--sglang-endpoint <local-endpoint>.Walkthrough
Mental model
The selected module is a locally managed process, not an SGLang plugin or registry entry. SGLang supplies only its loopback native-gRPC endpoint; the provider owns its arguments and runtime behavior.
Lifecycle and failure behavior
The module is imported only in the spawned child, preserving optional provider dependencies and avoiding parent-process import side effects. An import or
main(argv)contract failure terminates the required child and is handled by the existing watchdog; on normal shutdown SGLang stops the child before shutting down native gRPC.Boundaries
This is intentionally a same-process-namespace sidecar interface: wildcard SGLang binds are converted to loopback for the child. It does not add provider aliases, discovery, provider-specific arguments, or a registry.
Validation
.venv/bin/python -m pytest -q test/registered/unit/server_args/test_server_args.py::TestGrpcServerArgs— 16 passed.uvx pre-commit run --files python/sglang/srt/entrypoints/sidecar.py python/sglang/srt/entrypoints/http_server.py python/sglang/srt/server_args.py test/registered/unit/server_args/test_server_args.py— passed.--sidecar dynamo.sglang_sidecar.main: native gRPC discovery succeeded, Dynamo established eight gRPC connections, and servedsglang-sidecar-e2e.backend.generate.CI States
Latest PR Test (Base): 🚫 Run #29924027291
Latest PR Test (Extra): ❌ Run #29924026692