Repository navigation

[Perf] overlap: Stash the speculative relay in one launch - #48224

Open

heslami wants to merge 10 commits into

sgl-project:mainfrom

MarloweAI:up/spec-eagle-relay-stash

Open

[Perf] overlap: Stash the speculative relay in one launch#48224
heslami wants to merge 10 commits into
sgl-project:mainfrom
MarloweAI:up/spec-eagle-relay-stash

Commits on Nov 3, 2026
- [Perf] speculative: Draft top-1 in one launch
  Hassan Eslami
  committed
  28b18fe
- [Perf] speculative: Greedy verify of a top-1 chain in one launch
  Hassan Eslami
  committed
  f1ef12a
- [Perf] speculative: The verify launches write the accept lengths, the relayed lengths and the draft extend layout
  Hassan Eslami
  committed
  e538766
- [Perf] speculative: The first draft top-1 launch writes the chain's column 0
  Hassan Eslami
  committed
  ab69f79
- [Perf] cuda graph: EAGLE's target verify skips the re-copy of its pre-planned inputs
  Hassan Eslami
  committed
  2632d13
- [Perf] speculative: No positions restore after the draft graph's steps
  Hassan Eslami
  committed
  626b0d2
- [Perf] speculative: Fewer copies in the EAGLE draft graphs' input loads
  Hassan Eslami
  committed
  df00273
- [Perf] scheduler: No publish event that nothing waits on; the decode's KV-length copies only when allocating
  Hassan Eslami
  committed
  e14657f
- [AMD] speculative: Simulated acceptance runs its fused kernel on HIP
  Hassan Eslami
  committed
  bfb2243
- [Perf] overlap: Stash the speculative relay in one launch
  Hassan Eslami
  committed
  d0f4bd0