dflash piecewise cuda graphs support - #27468
Merged
Merged
Conversation
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Qiaolin-Yu
approved these changes
Jun 7, 2026
Qiaolin-Yu
left a comment
Qiaolin-Yu
left a comment
Collaborator
There was a problem hiding this comment.
lgtm. could you add a related test?
Collaborator
Author
|
/rerun-test test/registered/piecewise_cuda_graph/test_pcg_with_speculative_decoding_dflash.py |
Contributor
|
Results for 🚀 |
Oasis-Git
added a commit
to Oasis-Git/sglang
that referenced
this pull request
Jun 9, 2026
Three conflicts resolved + four-site API-migration port: 1. piecewise_cuda_graph_runner.py (deleted in HEAD, modified in main). Accept the deletion (cg-refactor's unified runner/prefill_cuda_graph_runner.py replaces it). Port main's sgl-project#27468 dflash-aux-hidden-states change to the new runner: capture_hidden_mode = FULL when spec_algorithm.is_dflash(), and use self.capture_hidden_mode (not NULL) in capture_prepare so dflash spec captures with the right variant. 2. frozen_kv_mtp_worker.py (deleted in main, modified in HEAD). Accept main's deletion (V1 -> V2 worker migration, sgl-project#27607). Apply the four cuda_graph_config API migration sites that cg-refactor had in V1 to V2 (frozen_kv_mtp_worker_v2.py): - backup_disable_cuda_graph -> backup_decode_mode (using new cuda_graph_config.decode.backend field) - server_args.disable_cuda_graph = True -> server_args.cuda_graph_config.decode.backend = Backend.DISABLED - matching restoration on draft_model_runner.server_args - init_cuda_graphs early-exit: self.server_args.disable_cuda_graph -> check_cuda_graph_backend(Phase.DECODE, Backend.DISABLED) 3. test/registered/cuda_graph/piecewise/ test_pcg_with_speculative_decoding_dflash.py (file location). Accept at the cg-refactor-renamed location (new test/registered/ cuda_graph/piecewise/ tree, not the legacy test/registered/piecewise_cuda_graph/). Audited the auto-merged code-side files (scheduler.py, schedule_batch.py, memory_pool.py, cuda_graph_buffer_registry.py, arg_groups/speculative_hook.py, spec_info.py, speculative_draft_runner.py) for unused-arg artifacts introduced by the merge. None found — added kwargs (e.g. mamba_indices on DSATokenToKVPool.get_cpu_copy/load_cpu_copy, padding_policy on positions/mrope_positions GraphSlots) are all referenced at the call sites and in the function bodies. Sanity-checked imports on touched files (frozen_kv_mtp_worker_v2, prefill_cuda_graph_runner) and AST-parsed cleanly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Chronostasys
pushed a commit
to MindLab-Research/sglang
that referenced
this pull request
Aug 24, 2026
jakki-amd
pushed a commit
to jakki-amd/sglang
that referenced
this pull request
Sep 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
split off from #23000 adds piecewise cuda graphs support to dflash speculative decoding
CI States
Latest PR Test (Base): 🚫 Run #27217067169
Latest PR Test (Extra): ❌ Run #27217065984