Skip to content

Capture resident KV aliases in recurrent CUDA Graphs - #96

Closed
0z5a wants to merge 81 commits into
ThinkFlowLab:mainfrom
0z5a:perf/alias-cuda-graph
Closed

0z5a wants to merge 81 commits into
ThinkFlowLab:mainfrom
0z5a:perf/alias-cuda-graph

Conversation

@0z5a

@0z5a 0z5a commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Incremental review: 0z5a/vllm-rlt#15, based on resident async aliases #95. The upstream diff includes the unmerged parent stack; use the fork PR for this change alone.

Purpose

Enable recurrent-core CUDA Graphs for the resident rectangular-alias path. Each graph entry owns stable depth tables and query-depth storage; replay refreshes the current request mapping and reads the live exit map through canonical-depth attention. Preserve the native graph-pool event ordering and update alias-readable history after submission. Finalized source positions remain immutable.

This is stacked on resident async aliases. Compact Graphs and synchronous alias Graphs remain unsupported.

Test Plan

  • Affected CPU alias/async/accounting checks: 29 passed, 12 CUDA skips.
  • All 60 finite CUDA arms pass at B1–128/C2B, with mixed depths, cancellation/reuse and mapped-host/H2D routing. The 24 native/alias Graph arms recorded 116 captures, 1,116 replays and zero fallbacks. Outputs/exits match native synchronous and all block credits/state slots return. Frozen source 4adada8 naturally exited 0 on RTX5090/Torch2.12.1+cu130.
  • Official Ouro1.4B delayed-policy diagnostic: all 36 arms at B1–128/C2B complete, with exact token sequences. Alias Graph has 143 request exit-sequence differences versus native Graph at B16/B32/B64. Recurrent work and actual batch distributions differ; no speed claim is made. Raw counters and frozen provenance.
  • Fixed logical replay at official B16/B32 reproduces native/alias complete request objects and every work counter; serialized comparison has 5,493 bitwise-equal hidden/gate/sample stages. All18 tiny replay arms and12 CUDA text-stop/reuse cases pass.
  • Frozen delayed-policy GSM8K100: native Graph63/100, alias Graph62/100, with15 token-sequence and43 exit-sequence differences. No quality-equivalence or speed claim. Full quality results and discordant answers.
  • Formal fixed-work and natural-scheduling performance remain pending.

Test Result

Fixed-policy E2E comparison Native Graph tokens/s Alias Graph tokens/s Speedup
Official checkpoint — — Not measured

Implementation and validation scope.

0z5a added 30 commits October 2, 2026 22:57
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
…ssion

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
…ive footprint

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
…ivals

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
0z5a added 12 commits October 5, 2026 05:47
…ting

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
…zation

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
…ting

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
0z5a added 16 commits October 5, 2026 08:36
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
…zation

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
…ting

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
…zation

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
…ting

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
@0z5a 0z5a closed this Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant