Skip to content

docs: SGLang gRPC workers can serve from Rust: SMG_SGLANG_SERVICER_IMPL=rust on the same entrypoint - #126

Merged
slin1237 merged 1 commit into
mainfrom
codex/nightly-docs-84b84025854cb500
Oct 8, 2026
Merged

slin1237 merged 1 commit into
mainfrom
codex/nightly-docs-84b84025854cb500

Conversation

@github-actions

@github-actions github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

What this PR does

How do I run an SGLang gRPC worker on the Rust servicer implementation — which environment variable selects it, what is the default, what must be installed, and which modes refuse it?

Why we need it

Source change: smg-project/smg@065807e

Commit 88 (#2750, 065807e1, 2026-10-04) added a Rust implementation of the SGLang gRPC servicer behind one environment variable, verified on current main. Selection: grpc_servicer/smg_grpc_servicer/sglang/rust.py:45 defines SERVICER_IMPL_ENV = "SMG_SGLANG_SERVICER_IMPL"; resolve_servicer_impl (rust.py:52-59) defaults to python and raises on values outside {python, rust}; smg_grpc_servicer/sglang/server.py:112-121 (serve_grpc, the entry behind python -m sglang.launch_server --smg-grpc-mode) hands the process to serve_rust before any Python gRPC machinery exists and warns that SGLang's HTTP sidecar (metrics/profiling endpoints) stays off on this path. serve_rust (rust.py:272-316) starts the Rust smg.servicer.SglangGrpcServer (binding listed in bindings/python/src/smg/servicer.py:26/:32; implementation crates/engine_servicer/src/sglang/) on the worker's host:port, launches the scheduler headless against a same-host ZMQ handshake, and supervises both; per the in-code comment at server.py:112-113 "the entrypoint and every ServerArgs flag stay the same, and the Router cannot tell them apart" — the gateway still connects over grpc://, so this is NOT the ZMQ direct backend of zmq-workers.md (grpc-workers.md:125 draws the same distinction for vLLM). Refusals: refuse_disaggregation (rust.py:252-269) fails startup under PD (disaggregation_mode set) or EPD (language_only/encoder_only); LoRA loading answers UNIMPLEMENTED (crates/engine_servicer/src/sglang/service.rs:17-30) and SubscribeKvEvents is relayed when the engine publishes KV events, UNIMPLEMENTED when they are off (crates/engine_servicer/src/sglang/mod.rs:140); grpc_servicer/README.md:295-313 states the user contract (needs the smg wheel; multimodal inputs and hidden states refused; Embed, FlushCache and profiling answer as on the Python servicer) — note rust.py's module docstring (lines 14-18) lags the README on those capabilities, so the writer must follow the README and the Rust code. No smg serve flag: bindings/python/src/smg/serve.py:826-837 scopes --servicer-impl to the vLLM gRPC servicer, parse_serve_args rejects it for other backends (serve.py:937-941), and only VllmWorkerLauncher.gpu_env manages a servicer env var (serve.py:321-334), so for SGLang the selection surface is the worker's environment. Missing docs: a repo-wide grep for SMG_SGLANG_SERVICER_IMPL and SglangGrpcServer finds nothing in src/lib/content; getting-started/grpc-workers.md's SGLang tab (:55-65) shows only the Python servicer start and the "Rust vLLM Servicer" section (:105-144) is explicitly vLLM-only; getting-started/index.md's SGLang worker recipe (:300-319) lacks the one-line Rust-servicer note its vLLM recipe carries at :298. Version framing: the commit postdates the v1.11.0 bump (commit 130, #2670, 2026-09-24), so present it as current-main behavior newer than v1.11.0, the style grpc-workers.md:107 already uses for the vLLM Rust servicer. One independent concern: one commit added one selection surface for one engine's gRPC servicer, documented where SGLang gRPC workers are started; the Rust path's full runtime capability matrix is the grpc-multimodal scan's charter and the edit should state limits only at the README's level. Dedup: no open or closed PR mentions SMG_SGLANG_SERVICER_IMPL or #2750 — merged #122 covers the different SGLang ZMQ direct backend (#2749), merged #105/#115 cover the vLLM servicer switch, closed-unmerged #87 covered SGLang gRPC worker shutdown (#2428, not recreated). No open PR touches grpc-workers.md or getting-started/index.md, and my other selected item edits only kv-events-cache-aware.md, so no files are shared. The TokenSpeed Rust servicer (commit 92, #2747, SMG_TOKENSPEED_SERVICER_IMPL) is a separate commit and concern, deferred because its canonical home is the same grpc-workers.md.

Scope: cli-configuration / sglang-rust-servicer-selection. Other concerns are deferred.

How to test

  • Passed the documentation path and size guard (under 1000 added plus deleted lines; no file-count limit).
  • Passed an independent accuracy and single-concern review.
  • Passed git diff --check and pnpm check and the production pnpm build.

Checklist

  • Tests added/updated (if applicable)
  • Docs updated (if applicable)
  • Commit includes a DCO sign-off

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Oct 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 592935b2-0997-4489-82b3-6f1a80407c80

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant