Repository navigation
docs: SGLang gRPC workers now shut down when their scheduler processes exit instead of lingering as healthy - #87
Closed
github-actions[bot] wants to merge 1 commit into
Closed
github-actions[bot] wants to merge 1 commit into
github-actions[bot] wants to merge 1 commit into
Conversation
Signed-off-by: XinyueZhang369 <zoeyzhang369@gmail.com>
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
This was referenced Oct 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
What happens to an SGLang worker started with --smg-grpc-mode when its scheduler processes crash or exit - does the servicer process keep running, and how does the worker come back?
Why we need it
Source change: smg-project/smg@f45d339
Commit 58 (#2428, merged 2026-09-22, before the v1.11.0 bump commit 23 that released smg-grpc-servicer 0.13.0) added grpc_servicer/smg_grpc_servicer/sglang/scheduler_watchdog.py, live on current main: watch_scheduler_processes polls every scheduler subprocess's exitcode once per second and, once ALL have exited, logs 'All scheduler processes exited ... Shutting down gRPC server' (error level when any exit code is non-zero) and sets the stop event; wait_for_scheduler_shutdown owns the watchdog task and is wired into the serve loop at grpc_servicer/smg_grpc_servicer/sglang/server.py:383, whose finally block marks the health servicer NOT_SERVING first so probes stop routing, then drains and stops the gRPC server - the servicer process exits. This matters because the health check alone never caught the idle case: grpc_servicer/smg_grpc_servicer/sglang/health_servicer.py:121 answers NOT_SERVING for a stuck scheduler only when requests are pending (>30s since last receive AND rid_to_state non-empty), so before #2428 an idle servicer with crashed schedulers kept answering SERVING and SMG kept the worker Ready until a request hit it; the process also had to be killed manually. Tests test_sglang_scheduler_watchdog.py pin the behavior. No docs page mentions it: src/lib/content/getting-started/grpc-workers.md (the page that starts SGLang gRPC workers) says nothing about servicer lifecycle on scheduler exit, and concepts/reliability/health-checks.md:82 only describes the gateway-side HealthCheck probe. This is one independent concern: one lifecycle behavior from one commit, documented with a short note on the one page that owns SGLang gRPC worker setup and troubleshooting (a restarted servicer on the same address rejoins via SMG's continued probing). Runtime routing/health thresholds stay with the routing-workers scan.
Scope: grpc-workers / sglang-scheduler-exit-shutdown. Other concerns are deferred.
How to test
git diff --checkandpnpm checkand the productionpnpm build.Checklist