Skip to content

[Fix] scheduler: throttle polling loop when scheduler is paused - #46307

Open
PratikCreates wants to merge 1 commit into
sgl-project:mainfrom
PratikCreates:fix/pause-scheduler-busy-loop
Open

PratikCreates wants to merge 1 commit into
sgl-project:mainfrom
PratikCreates:fix/pause-scheduler-busy-loop

Conversation

@PratikCreates

@PratikCreates PratikCreates commented Oct 24, 2026 •

Copy link
Copy Markdown

Motivation

When the scheduler is paused (self._engine_paused = True), the event loops repeatedly poll non-blocking sockets and record metrics in a tight loop without waiting, which can spin CPU cores during pause states.

Modifications

  • In _record_scheduler_state_for_paused_engine, call maybe_sleep_on_idle() if an idle sleeper is configured.
  • Fall back to sleeping 5ms when an idle sleeper is not available.
  • Added est_paused_engine_waits_or_throttles to est/registered/unit/managers/test_scheduler_pause_generation.py.

Checklist

  • Add unit tests according to the Run and add unit tests.

CI States

Latest PR Test (Base): ❌ Run #41350119225
Latest PR Test (Extra): ❌ Run #41350119211
Latest PR Test (AMD ROCm 10): ❌ Run #41350119124

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

When the scheduler is paused, the event loop continuously polled non-blocking sockets and recorded metrics in a tight loop without waiting.

Call maybe_sleep_on_idle if an idle sleeper is available, or fall back to sleeping 5ms to avoid spinning CPU cores while paused. Added unit test.

Signed-off-by: PratikCreates <139613382+PratikCreates@users.noreply.github.com>
@PratikCreates
PratikCreates force-pushed the fix/pause-scheduler-busy-loop branch from 3876cd4 to 6b9b4c9 Compare November 1, 2026 21:04

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant