Skip to content

[Dsv4.1] allow decoder bounded replay with breakable prefill graphs - #43440

Closed
Oasis-Git wants to merge 1 commit into
sgl-project:mainfrom
Oasis-Git:fix/dsv41-bcg-decoder-bounded
Closed

Oasis-Git wants to merge 1 commit into
sgl-project:mainfrom
Oasis-Git:fix/dsv41-bcg-decoder-bounded

Conversation

@Oasis-Git

@Oasis-Git Oasis-Git commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Keep captured prefills on full rows, clear stale eager-tail metadata on capture and replay, and only trim model/logits inputs when the current batch carries tail metadata. Larger eager prefills retain decoder bounded replay; DP attention and other prefill graph backends remain unsupported.

Add six CPU regression tests covering configuration guards, eager/graph metadata transitions and logits row selection. All six pass against the rebased source and all pre-commit checks pass. Earlier lab GPU/quality/AgentX runs provide context only; this rebased branch has not received a new GPU end-to-end run.

Motivation

Modifications

Accuracy Tests

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ❌ Run #38000450710
Latest PR Test (Extra): ❌ Run #38000450232
Latest PR Test (AMD ROCm 10): ❌ Run #38000450544

Keep captured prefills on full rows, clear stale eager-tail metadata on capture and replay, and only trim model/logits inputs when the current batch carries tail metadata. Larger eager prefills retain decoder bounded replay; DP attention and other prefill graph backends remain unsupported.

Add six CPU regression tests covering configuration guards, eager/graph metadata transitions and logits row selection. All six pass against the rebased source and all pre-commit checks pass. Earlier lab GPU/quality/AgentX runs provide context only; this rebased branch has not received a new GPU end-to-end run.
@Oasis-Git

Copy link
Copy Markdown
Collaborator Author

close with a better implementation #43736

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deepseek run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant