Repository navigation
Conversation
…trap # Conflicts: # python/sglang/srt/disaggregation/common/conn.py # rust/sglang-server/src/api_server/disaggregation/bootstrap.rs
…trap # Conflicts: # rust/sglang-server/src/api_server/disaggregation/bootstrap.rs
This was referenced Sep 24, 2026
15 of 40 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Target-only prefill does not initialize the draft model's prefix KV or proposal state. Decode-side EAGLE therefore needs real initialization rather than accepting absent metadata or synthesizing probabilities.
Dependency
Depends on #40953.
This stacked PR targets
main, so its full diff includes that prerequisite. Review the decode-bootstrap increment separately; it will need rebasing after the prerequisite merges. It does not depend on the workload-routing PR.Modifications
--disaggregation-decode-draft-bootstrapand an explicit prefix-token budget.pd_draft_bootstrap_tokens, without altering prefill cache-hit accounting.This is an experimental correctness fallback, not a throughput optimization: it repeats full-prefix target and draft compute. Supported scope is single-layer EAGLE, TP-only decode, supported DeepSeek/GLM MoE text architectures, Mooncake, and Python egress. Decode prefix caching, HiCache, offload, HiSparse, constrained decoding, embeddings and multimodal requests are excluded. Rust egress is rejected for this mode because it does not expose replay accounting.
Accuracy Tests
git diff --check.The manual README documents the full-environment test commands and outstanding GPU validation matrix.
Speed Tests and Profiling
No speedup or benchmark result is claimed. Account for the replay counter separately in computed-token metrics. No internal model fixtures, cluster configs, logs or results are included.
Checklist
CI States
Latest PR Test (Base): ❌ Run #35906910682
Latest PR Test (Extra): ❌ Run #35906910092
Latest PR Test (AMD ROCm 10): ❌ Run #35906910780