Skip to content

[Spec] Publish DFLASH verify read-done event for fine-grained WAR barrier - #29541

Merged
hnyls2002 merged 2 commits into
mainfrom
lsyin/dflash-verify-read-done
Jun 29, 2026
Merged

hnyls2002 merged 2 commits into
mainfrom
lsyin/dflash-verify-read-done

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator

What

Publish the WAR read_done event from the DFLASH target-verify forward so the scheduler's global _apply_war_barrier takes the fine-grained wait_event(read_done) fast path instead of the coarse wait_stream(forward_stream) fallback.

Why it is valid

The target-verify cuda graph builds its page table in load_batch (pre-replay) and then replays reading only that static snapshot. So the forward finishes reading the shared req_to_token / SWA buffers before replay — read_done is recorded at exactly that point, so it is a sound WAR signal.

WAR ordering

The over-alloc write to req_to_token (plan stream) stays ordered after the previous forward's read through this chain:

read_done  (recorded on the forward stream, after req_to_token is read)
   │  decode_cuda_graph_runner: war_fastpath_read_done_event = read_done
   ▼
schedule_stream.wait_event(read_done)         scheduler.py:1521  (_apply_war_barrier, called at loop top :1578)
   │
   ▼
plan_stream.wait_stream(schedule_stream)      dflash_info_v2.py:188  (prepare_for_decode)
   │
   ▼
over-alloc writes req_to_token (plan stream)  dflash_info_v2.py:218

prepare_for_decode already does plan_stream.wait_stream(schedule_stream) (originally to see the scheduler's filter/merge writes), so the plan-stream over-alloc write inherits the read_done dependency transitively — no explicit plan-stream wait is added. The barrier now waits only until req_to_token has been read (start of the verify forward) instead of the whole forward, recovering the overlap window the coarse fallback serialized.

Dependencies

Builds on #29556 (merged) — removed verify_done and the dflash _war_barrier_enabled opt-out, routing dflash through _apply_war_barrier. Independent of #29343 (fa3).


CI States

Latest PR Test (Base): 🚫 Run #28342455012
Latest PR Test (Extra): ❌ Run #28342454942

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@hnyls2002
hnyls2002 force-pushed the lsyin/dflash-verify-read-done branch from 1dd9390 to 0ddcba7 Compare June 28, 2026 03:40
@github-actions github-actions Bot added the blackwell SM100/SM120 label Jun 28, 2026
@hnyls2002
hnyls2002 force-pushed the lsyin/dflash-verify-read-done branch from 0ddcba7 to 9df2fa9 Compare June 28, 2026 05:52
@hnyls2002
hnyls2002 force-pushed the lsyin/fa3-gpu-only-seq-lens branch from 788ee42 to 2307cb2 Compare June 28, 2026 07:15
@hnyls2002
hnyls2002 force-pushed the lsyin/dflash-verify-read-done branch 2 times, most recently from f6051be to b0b8aab Compare June 28, 2026 08:01
@hnyls2002
hnyls2002 force-pushed the lsyin/fa3-gpu-only-seq-lens branch 3 times, most recently from 7888a48 to 25f4857 Compare June 28, 2026 23:40
@hnyls2002
hnyls2002 force-pushed the lsyin/dflash-verify-read-done branch from b0b8aab to 0f9c94c Compare June 28, 2026 23:55
@hnyls2002
hnyls2002 force-pushed the lsyin/dflash-verify-read-done branch from 0f9c94c to 1d369b7 Compare June 29, 2026 00:00
@hnyls2002

hnyls2002 commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test registered/spec/dflash/test_dflash.py registered/cuda_graph/piecewise/test_pcg_with_speculative_decoding_dflash.py registered/spec/test_gemma4_dflash_31b_extra.py (latest main)

@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Results for /rerun-test registered/spec/dflash/test_dflash.py registered/cuda_graph/piecewise/test_pcg_with_speculative_decoding_dflash.py registered/spec/test_gemma4_dflash_31b_extra.py:

🚀 1-gpu-5090 (2 tests): ✅ View workflow run

cd test/ && python3 registered/spec/dflash/test_dflash.py
cd test/ && python3 registered/cuda_graph/piecewise/test_pcg_with_speculative_decoding_dflash.py

🚀 2-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/spec/test_gemma4_dflash_31b_extra.py

@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Results for /rerun-test registered/spec/dflash/test_dflash.py registered/cuda_graph/piecewise/test_pcg_with_speculative_decoding_dflash.py registered/spec/test_gemma4_dflash_31b_extra.py (latest main):

🚀 1-gpu-5090 (2 tests): ✅ View workflow run

cd test/ && python3 registered/spec/dflash/test_dflash.py
cd test/ && python3 registered/cuda_graph/piecewise/test_pcg_with_speculative_decoding_dflash.py

🚀 2-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/spec/test_gemma4_dflash_31b_extra.py

(latest: No test file found matching (latest under test/registered/ or python/sglang/multimodal_gen/test/.

main): No test file found matching main) under test/registered/ or python/sglang/multimodal_gen/test/.

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@hnyls2002
hnyls2002 merged commit be17475 into main Jun 29, 2026
193 of 236 checks passed
@hnyls2002
hnyls2002 deleted the lsyin/dflash-verify-read-done branch June 29, 2026 01:51
smartssw pushed a commit to smartssw/sglang that referenced this pull request Jul 6, 2026
Chronostasys pushed a commit to MindLab-Research/sglang that referenced this pull request Aug 24, 2026
jakki-amd pushed a commit to jakki-amd/sglang that referenced this pull request Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant