[zhtshr]

Motivation

When the scheduler becomes idle, it refreshes its local and Prometheus statistics without publishing KvMetrics. Consumers such as Dynamo can retain the last busy-batch KV usage and continue rejecting requests after the engine is idle.

Modifications

• Publish refreshed KV metrics from _maybe_log_idle_metrics(), preserving the existing metrics gate and idle reporting interval.
• Add regression coverage for idle updates in regular, prefill, and decode modes, including nonzero remaining KV usage, rate limiting, and disabled metrics.

Accuracy Tests

• All 54 related CPU unit tests pass. The added regression tests fail without the fix.
• Real-GPU Qwen3-0.6B 1P1D validation with a 60% KV threshold: the baseline completes the first request, then rejects three requests while actual prefill KV usage is 0% and reported usage remains 79%. With the fix, all four requests complete and idle KV metrics return to 0%.
• Three additional busy/idle cycles confirm that genuine busy-worker rejection still works and admission recovers after idle. Every successful request generates 256 tokens.

Speed Tests and Profiling

N/A: no model or kernel changes. The GPU checks above validate admission recovery.

Checklist

• Format code with the repository's full pre-commit checks.
• Add focused regression tests.
• Documentation: N/A; no user-facing interface changes.
• Accuracy/speed benchmarks: N/A; functional regression results are provided above.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): ✅ Run #41420374872
Latest PR Test (Extra): ❌ Run #41420375008
Latest PR Test (AMD ROCm 10): ✅ Run #41420374992

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

[ishandhanani]

/tag-and-rerun-ci

[ishandhanani]

Round 1 AI review

Automated review (round 1 of up to 2 before a human review). Findings may be wrong; reply on a thread to dispute one.

This PR makes the idle-metrics path call kv_events_publisher.emit_kv_metrics() right after log_stats(). Before, a router that reads KvMetrics kept seeing the last batch's kv_active_blocks, request_active_slots and num_requests_waiting after the scheduler went idle. The one-line production change is correct and matches the two batch-report call sites. It keeps the current_scheduler_metrics_enabled gate and the 30 s rate limit, and emit_kv_metrics returns early when KV events are off. Two CPU unit tests fail at the head SHA because their test setup does not match current main; both errors appear in the base-a-test-cpu (1) and (4) logs. The Lint failure is not from this PR: it is an F821 in xpu_graph_runner.py.

New findings: 2 major

The next AI round runs after new commits are pushed.

[ishandhanani]

Round 2 AI review

Automated review (round 2 of up to 2 before a human review). Findings may be wrong; reply on a thread to dispute one.

This PR makes the scheduler's idle-metrics path call kv_events_publisher.emit_kv_metrics(). Without it, a router reading KvMetrics keeps the last batch's active-block, active-slot and waiting counts while the scheduler is idle. The new commit c898c2c890 fixes the test setup only; the production change is the same one-line call reviewed in round 1. Both round-1 findings are fixed, and all 18 base-a-test-cpu shards now pass at c898c2c890. I found no new issues.

New findings: none
Earlier findings: 2 fixed, 0 no longer apply, 0 still open

AI review is done; a human reviewer takes it from here.

[ishandhanani]

LGTM

[ishandhanani]

/rerun-test registered/unit/observability/test_forward_pass_metrics.py registered/unit/managers/test_disagg_idle_step_counters.py registered/unit/managers/test_scheduler_on_idle_load.py

[github-actions]

🚀 ubuntu-latest (3 tests): ✅ View workflow run

cd test/ && python3 registered/unit/observability/test_forward_pass_metrics.py
cd test/ && python3 registered/unit/managers/test_disagg_idle_step_counters.py
cd test/ && python3 registered/unit/managers/test_scheduler_on_idle_load.py