Skip to content

[Spec] Avoid multi-layer EAGLE SWA scheduler synchronization - #35378

Closed
paulzhang-tm wants to merge 1 commit into
sgl-project:mainfrom
paulzhang-tm:codex/avoid-mtp-swa-scheduler-sync
Closed

paulzhang-tm wants to merge 1 commit into
sgl-project:mainfrom
paulzhang-tm:codex/avoid-mtp-swa-scheduler-sync

Conversation

@paulzhang-tm

@paulzhang-tm paulzhang-tm commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Motivation

Paged sliding-window KV eviction performs a data-dependent uniqueness pass for request segments that are already complete and page-aligned. Multi-layer EAGLE also leaves the scheduler without the final draft graph's shared-read completion event, forcing a broader stream synchronization before the shared buffers can be reused.

Modifications

  • Add a fixed-shape SWA segment-free path for page-aligned request prefixes, with the existing token path retained for page size 1 and unified allocation.
  • Route sliding-window eviction through that path when the allocator supports it.
  • Publish shared_read_done_event after the final multi-layer draft graph step, including the single-graph replay path.
  • Point the worker's last shared-read runner at the final draft runner.
  • Add allocator and CPU event-publication regression tests.

Accuracy Tests

No model-output behavior changes. The relevant unit suites pass:

14 SWA allocator tests passed (plus 15 subtests)
14 SWA eviction-boundary tests passed
2 shared-read event tests passed

Speed Tests and Profiling

Not benchmarked independently. The allocator change removes a device-side uniqueness operation from complete page-aligned SWA segments, and the event publication lets the scheduler use its event-scoped shared-read barrier for multi-layer EAGLE.

Checklist

-Robot


CI States

Latest PR Test (Base): ❌ Run #32182552708
Latest PR Test (Extra): ❌ Run #32182552201

Free page-aligned sliding-window KV segments without GPU uniqueness checks and publish the final draft graph's shared-read completion event for scheduler overlap.
@paulzhang-tm

Copy link
Copy Markdown
Contributor Author

Superseded by #36751 for the paged SWA allocator change and #36752 for the multi-layer EAGLE shared-read event change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant