Skip to content

feat(sglang): add managed Python sidecar entrypoint - #11796

Closed
connorcarpenter15 wants to merge 2 commits into
mainfrom
feat/sglang-python-sidecar-entrypoint
Closed

connorcarpenter15 wants to merge 2 commits into
mainfrom
feat/sglang-python-sidecar-entrypoint

Conversation

@connorcarpenter15

@connorcarpenter15 connorcarpenter15 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replace the top-level dynamo.sglang_sidecar module with the SGLang-scoped dynamo.sglang.sidecar entry point.
  • Move the native launcher to the private dynamo._core.backend._run_sglang_sidecar(argv) binding.
  • Preserve ordinary Python argument semantics by prepending the executable name before clap parses the arguments.
  • Release the GIL while bootstrapping and running Dynamo's unified worker lifecycle.
  • Configure Dynamo logging only when main() runs, not when the module is imported.
  • Add binding and Python launcher coverage.

Motivation

SGLang's module-based sidecar interface needs an importable entry point that composes the native SGLang engine adapter with Dynamo's unified backend lifecycle. Keeping this under dynamo.sglang makes the ownership and invocation clearer while retaining the released wheel as the only required installation.

python3 -m sglang.launch_server \
    <args> \
    --grpc-port 30001 \
    --sidecar dynamo.sglang.sidecar

Scope

This extracts the Python entry-point portion of #11636. It intentionally leaves the standalone executable changes and the lib/ directory organization to separate PRs. The Rust crate remains at lib/sglang-sidecar.

Validation

  • cargo fmt --all -- --check
  • cargo metadata --manifest-path lib/bindings/python/Cargo.toml --no-deps --format-version 1
  • ruff check on the changed Python tests and launcher
  • git diff --check

A full local Cargo test build could not complete because the workspace filesystem ran out of space while compiling Dynamo's transitive dependencies; draft PR CI will provide the full build and test result.

Summary by CodeRabbit

  • New Features

    • Added a dedicated SGLang sidecar launcher that supports command-line arguments and uses the native backend runner.
    • Updated the Python interface to expose the launcher through the backend module.
  • Removed

    • Removed the previous standalone SGLang sidecar entry point.
  • Tests

    • Added coverage for argument forwarding, default process arguments, logging behavior, and backend interface availability.

Signed-off-by: Connor Carpenter <connorc@nvidia.com>
@github-actions github-actions Bot added feat backend::sglang Relates to the sglang backend actions labels Jul 16, 2026
@datadog-official

datadog-official Bot commented Jul 16, 2026

Copy link
Copy Markdown

Pipelines

⚠️ Warnings

🚦 7 Pipeline jobs failed

DCO Commenter | dco-comment   View in Datadog   GitHub Actions

Fern Docs | changed-files   View in Datadog   GitHub Actions

PR | backend-status-check   View in Datadog   GitHub Actions

View all 7 failed jobs.

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: fff00f1 | Docs | Give us feedback!

@connorcarpenter15
connorcarpenter15 marked this pull request as ready for review July 16, 2026 21:36
@connorcarpenter15
connorcarpenter15 requested review from a team as code owners July 16, 2026 21:36

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

The SGLang sidecar launcher now runs through a backend-scoped Rust binding and a new Python module. The former top-level entrypoint is removed, type stubs and tests are updated, and ownership and CI path mappings exclude the old sidecar directory.

SGLang sidecar launcher

Layer / File(s) Summary
Backend launcher binding
lib/bindings/python/rust/backend.rs, lib/bindings/python/rust/llm/entrypoint.rs, lib/bindings/python/rust/lib.rs, lib/bindings/python/src/dynamo/_core.pyi, components/src/dynamo/common/backend/tests/test_backend_bindings.py
Adds backend._run_sglang_sidecar, removes the former top-level export, updates the type stub, and verifies the backend attribute is callable.
Python sidecar entrypoint and tests
components/src/dynamo/sglang/sidecar.py, components/src/dynamo/sglang/tests/test_sidecar.py
Adds logging-aware argument forwarding through dynamo.sglang.sidecar and tests explicit arguments, process arguments, and import behavior.
SGLang path routing updates
.github/codeowners/areas.yaml, .github/filters.yaml, CODEOWNERS
Removes the old sglang_sidecar path from backend ownership and SGLang CI filtering mappings.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is detailed but does not follow the required template and omits the mandatory Related Issues section and reviewer-start guidance. Add the required Overview, Details, Where should reviewer start?, and Related Issues sections, and include either the linked issue number or the no-issue confirmation.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the main change: adding a managed Python SGLang sidecar entrypoint.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
lib/bindings/python/rust/llm/entrypoint.rs (1)

951-956: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Reuse the crate-level to_pyerr helper
lib/bindings/python/rust/llm/entrypoint.rs:951 duplicates the shared crate::to_pyerr already defined in lib/bindings/python/rust/lib.rs. Use the shared helper here instead of keeping a second conversion path.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/bindings/python/rust/llm/entrypoint.rs` around lines 951 - 956, Remove
the local to_pyerr function in entrypoint.rs and reuse the existing crate-level
crate::to_pyerr helper for all corresponding error conversions, preserving the
current PyErr behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@lib/bindings/python/rust/llm/entrypoint.rs`:
- Around line 951-956: Remove the local to_pyerr function in entrypoint.rs and
reuse the existing crate-level crate::to_pyerr helper for all corresponding
error conversions, preserving the current PyErr behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ae44fcb0-2e5f-46df-985e-3852793b4c31

📥 Commits

Reviewing files that changed from the base of the PR and between 01d2cb2 and f827205.

📒 Files selected for processing (13)
  • .github/codeowners/areas.yaml
  • .github/filters.yaml
  • CODEOWNERS
  • components/src/dynamo/common/backend/tests/test_backend_bindings.py
  • components/src/dynamo/sglang/sidecar.py
  • components/src/dynamo/sglang/tests/test_sidecar.py
  • components/src/dynamo/sglang_sidecar/__init__.py
  • components/src/dynamo/sglang_sidecar/__main__.py
  • components/src/dynamo/sglang_sidecar/main.py
  • lib/bindings/python/rust/backend.rs
  • lib/bindings/python/rust/lib.rs
  • lib/bindings/python/rust/llm/entrypoint.rs
  • lib/bindings/python/src/dynamo/_core.pyi
💤 Files with no reviewable changes (7)
  • components/src/dynamo/sglang_sidecar/main.py
  • components/src/dynamo/sglang_sidecar/init.py
  • .github/filters.yaml
  • CODEOWNERS
  • components/src/dynamo/sglang_sidecar/main.py
  • .github/codeowners/areas.yaml
  • lib/bindings/python/rust/lib.rs

Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

actions backend::sglang Relates to the sglang backend feat size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant