Skip to content

docs: SGLang gRPC workers now shut down when their scheduler processes exit instead of lingering as healthy - #87

Closed
github-actions[bot] wants to merge 1 commit into
mainfrom
codex/nightly-docs-6a67953ef68c120f
Closed

github-actions[bot] wants to merge 1 commit into
mainfrom
codex/nightly-docs-6a67953ef68c120f

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

What this PR does

What happens to an SGLang worker started with --smg-grpc-mode when its scheduler processes crash or exit - does the servicer process keep running, and how does the worker come back?

Why we need it

Source change: smg-project/smg@f45d339

Commit 58 (#2428, merged 2026-09-22, before the v1.11.0 bump commit 23 that released smg-grpc-servicer 0.13.0) added grpc_servicer/smg_grpc_servicer/sglang/scheduler_watchdog.py, live on current main: watch_scheduler_processes polls every scheduler subprocess's exitcode once per second and, once ALL have exited, logs 'All scheduler processes exited ... Shutting down gRPC server' (error level when any exit code is non-zero) and sets the stop event; wait_for_scheduler_shutdown owns the watchdog task and is wired into the serve loop at grpc_servicer/smg_grpc_servicer/sglang/server.py:383, whose finally block marks the health servicer NOT_SERVING first so probes stop routing, then drains and stops the gRPC server - the servicer process exits. This matters because the health check alone never caught the idle case: grpc_servicer/smg_grpc_servicer/sglang/health_servicer.py:121 answers NOT_SERVING for a stuck scheduler only when requests are pending (>30s since last receive AND rid_to_state non-empty), so before #2428 an idle servicer with crashed schedulers kept answering SERVING and SMG kept the worker Ready until a request hit it; the process also had to be killed manually. Tests test_sglang_scheduler_watchdog.py pin the behavior. No docs page mentions it: src/lib/content/getting-started/grpc-workers.md (the page that starts SGLang gRPC workers) says nothing about servicer lifecycle on scheduler exit, and concepts/reliability/health-checks.md:82 only describes the gateway-side HealthCheck probe. This is one independent concern: one lifecycle behavior from one commit, documented with a short note on the one page that owns SGLang gRPC worker setup and troubleshooting (a restarted servicer on the same address rejoins via SMG's continued probing). Runtime routing/health thresholds stay with the routing-workers scan.

Scope: grpc-workers / sglang-scheduler-exit-shutdown. Other concerns are deferred.

How to test

  • Passed the documentation path and size guard (under 1000 added plus deleted lines; no file-count limit).
  • Passed an independent accuracy and single-concern review.
  • Passed git diff --check and pnpm check and the production pnpm build.

Checklist

  • Tests added/updated (if applicable)
  • Docs updated (if applicable)
  • Commit signed off by XinyueZhang369 zoeyzhang369@gmail.com

Signed-off-by: XinyueZhang369 <zoeyzhang369@gmail.com>
@coderabbitai

coderabbitai Bot commented Sep 30, 2026

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 6b2b8435-f49d-421e-b8f2-782d146b2237

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants