[Rust] Gate health on startup warmup completion

[nvpohanh]

[by Codex]

Summary

• Align Rust frontend readiness with the Python frontend: a successful health response must mean startup warmup has completed.
• Keep Rust /health and /health_generate at 503 until startup warmup has completed successfully.
• Mark the main process's language, embedding, or multimodal warmup request and flip Rust readiness only after that request returns 2xx.
• Preserve --skip-server-warmup: when it is explicitly set, Rust starts ready.
• Leave test_openai_completion_rust.py unchanged; radix cache and normal server warmup remain enabled.

CI impact

This fixes a recurring flake in test/registered/openai_server/basic/test_openai_completion_rust.py that has broken pre-merge CI for many unrelated changes. I found the same failure in at least eight pre-merge runs across seven PRs: #28403, #30315, #33068, #35599, #37284, #37601 (twice), and #37820. The test compares Python and Rust frontend logprobs exactly, so the readiness race could fail an otherwise healthy PR without any relevant code changes.

Error and root cause

The Python frontend keeps both health endpoints at 503 while ServerStatus is Starting, and changes the status only after startup warmup succeeds. The Rust listener is intentionally bound before the main process runs _execute_server_warmup, because the warmup path first queries /model_info. However, the Rust health handler previously submitted its own one-token scheduler probe immediately and returned 200 when that probe made progress. Rust could therefore advertise readiness earlier than Python: popen_launch_server could observe a healthy server and send the parity test's completion while the real eight-token startup warmup was still running.

With radix cache enabled, this race changed which request populated or reused the shared prompt prefix. The Python launch commonly evaluated the completion with #new-token: 3, #cached-token: 2/3, while the raced Rust launch evaluated the full prompt with #new-token: 5/6, #cached-token: 0, or co-batched it with warmup. The output tokens were the same, but the different forward-pass shape could produce bitwise-different logprobs on the CI GPU. Radix cache exposed the startup race; it was not the underlying bug.

This behavior originated with the native Rust health handlers in #32876. The parity test was added in #33103. The earlier flake caused by default prefill CUDA graphs (#33352) was a separate issue and was fixed by #34146.

Why this works

The main process adds an internal marker header only to the startup warmup request. Rust middleware recognizes that marker only on the three actual warmup routes (/generate, /encode, and /v1/chat/completions) and records readiness after a successful response. Both health modes check the same atomic readiness state before returning 200 or running a generation probe. Therefore Rust now follows the Python frontend's ServerStatus.Starting contract: health remains 503 during warmup and a health success implies that startup warmup has completed.

If warmup fails, readiness remains false. If warmup is explicitly skipped, the launch-time flag initializes readiness true.

Validation

• Mandatory full pre-commit command passed, including Rust workspace clippy and rustfmt.
• Focused Rust tests passed for readiness transitions and pre-warmup health returning 503.
• ComputeLab L40S: unchanged test_openai_completion_rust.py passed with normal warmup and radix cache enabled (1 passed, 421.484 s).
• The Rust run log showed startup warmup first (POST /generate 200, 6 new / 0 cached), then health (GET /health_generate 200), then the tested completion (3 new / 2 cached), matching the Python request shape.

Failure being addressed: https://github.com/sgl-project/sglang/actions/runs/33831875719/job/100902930637?pr=35599

---

CI States

Latest PR Test (Base): ✅ Run #34100216762
Latest PR Test (Extra): ✅ Run #34414379326
Latest PR Test (AMD ROCm 10): ❌ Run #34100215649

[nvpohanh]

cc @rainj-me @mrain

[nvpohanh]

/rerun-tests test/registered/openai_server/basic/test_openai_completion_rust.py

[github-actions]

Results for /rerun-tests test/registered/openai_server/basic/test_openai_completion_rust.py:

🚀 1-gpu-5090 (1 test): ✅ View workflow run

cd test/ && python3 registered/openai_server/basic/test_openai_completion_rust.py

[nvpohanh]

/tag-and-rerun-ci

[nvpohanh]

/rerun-failed-ci

[nvpohanh]

All NV pipelines have passed