Repository navigation
docs: SGLang gRPC workers can serve from Rust: SMG_SGLANG_SERVICER_IMPL=rust on the same entrypoint - #126
Merged
Conversation
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
slin1237
marked this pull request as ready for review
October 8, 2026 22:01
slin1237
requested review from
CatherineSue,
key4ng and
slin1237
as code owners
October 8, 2026 22:01
This was referenced Oct 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
How do I run an SGLang gRPC worker on the Rust servicer implementation — which environment variable selects it, what is the default, what must be installed, and which modes refuse it?
Why we need it
Source change: smg-project/smg@065807e
Commit 88 (#2750, 065807e1, 2026-10-04) added a Rust implementation of the SGLang gRPC servicer behind one environment variable, verified on current main. Selection: grpc_servicer/smg_grpc_servicer/sglang/rust.py:45 defines SERVICER_IMPL_ENV = "SMG_SGLANG_SERVICER_IMPL"; resolve_servicer_impl (rust.py:52-59) defaults to python and raises on values outside {python, rust}; smg_grpc_servicer/sglang/server.py:112-121 (serve_grpc, the entry behind
python -m sglang.launch_server --smg-grpc-mode) hands the process to serve_rust before any Python gRPC machinery exists and warns that SGLang's HTTP sidecar (metrics/profiling endpoints) stays off on this path. serve_rust (rust.py:272-316) starts the Rust smg.servicer.SglangGrpcServer (binding listed in bindings/python/src/smg/servicer.py:26/:32; implementation crates/engine_servicer/src/sglang/) on the worker's host:port, launches the scheduler headless against a same-host ZMQ handshake, and supervises both; per the in-code comment at server.py:112-113 "the entrypoint and every ServerArgs flag stay the same, and the Router cannot tell them apart" — the gateway still connects over grpc://, so this is NOT the ZMQ direct backend of zmq-workers.md (grpc-workers.md:125 draws the same distinction for vLLM). Refusals: refuse_disaggregation (rust.py:252-269) fails startup under PD (disaggregation_mode set) or EPD (language_only/encoder_only); LoRA loading answers UNIMPLEMENTED (crates/engine_servicer/src/sglang/service.rs:17-30) and SubscribeKvEvents is relayed when the engine publishes KV events, UNIMPLEMENTED when they are off (crates/engine_servicer/src/sglang/mod.rs:140); grpc_servicer/README.md:295-313 states the user contract (needs the smg wheel; multimodal inputs and hidden states refused; Embed, FlushCache and profiling answer as on the Python servicer) — note rust.py's module docstring (lines 14-18) lags the README on those capabilities, so the writer must follow the README and the Rust code. No smg serve flag: bindings/python/src/smg/serve.py:826-837 scopes --servicer-impl to the vLLM gRPC servicer, parse_serve_args rejects it for other backends (serve.py:937-941), and only VllmWorkerLauncher.gpu_env manages a servicer env var (serve.py:321-334), so for SGLang the selection surface is the worker's environment. Missing docs: a repo-wide grep for SMG_SGLANG_SERVICER_IMPL and SglangGrpcServer finds nothing in src/lib/content; getting-started/grpc-workers.md's SGLang tab (:55-65) shows only the Python servicer start and the "Rust vLLM Servicer" section (:105-144) is explicitly vLLM-only; getting-started/index.md's SGLang worker recipe (:300-319) lacks the one-line Rust-servicer note its vLLM recipe carries at :298. Version framing: the commit postdates the v1.11.0 bump (commit 130, #2670, 2026-09-24), so present it as current-main behavior newer than v1.11.0, the style grpc-workers.md:107 already uses for the vLLM Rust servicer. One independent concern: one commit added one selection surface for one engine's gRPC servicer, documented where SGLang gRPC workers are started; the Rust path's full runtime capability matrix is the grpc-multimodal scan's charter and the edit should state limits only at the README's level. Dedup: no open or closed PR mentions SMG_SGLANG_SERVICER_IMPL or #2750 — merged #122 covers the different SGLang ZMQ direct backend (#2749), merged #105/#115 cover the vLLM servicer switch, closed-unmerged #87 covered SGLang gRPC worker shutdown (#2428, not recreated). No open PR touches grpc-workers.md or getting-started/index.md, and my other selected item edits only kv-events-cache-aware.md, so no files are shared. The TokenSpeed Rust servicer (commit 92, #2747, SMG_TOKENSPEED_SERVICER_IMPL) is a separate commit and concern, deferred because its canonical home is the same grpc-workers.md.Scope: cli-configuration / sglang-rust-servicer-selection. Other concerns are deferred.
How to test
git diff --checkandpnpm checkand the productionpnpm build.Checklist