Repository navigation
Conversation
|
2ms |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is a new PR against
a5-k3-0828that carries the K3 KDA/DSparkhot-path port from #8 and adds the dense/static follow-up optimizations.
lower_boundhandling.[B, T, C]causal-conv path for dense/statictarget verify (
SGLANG_NPU_KDA_DENSE_CONV3D=1).query_start_loc; zero-length graph-padding requests are mapped to-1and the same tensor is shared by causal-conv and KDA recurrent.
including B1, while preserving TBO and context-parallel alignment.
separate dense-only onorm experiment was not kept because it had no
measurable graph gain.
metadata, plus a reproducible full-model TP32/DP1 launcher.
The formatting-only baseline fixes are isolated in commit
313acfa5fandmatch the A5 branch's
pre-commit run --all-filesrequirements.Paired kernel PR
Requires zhaozx-cn/sgl-kernel-npu#3
from branch
McZyWu:a5-k3-0828-dense-static-opt-20260903.The framework passes
-1for graph-padding KDA state rows. The pairedkernel skips recurrence for that sentinel, zeroes the dense output row, and
does not touch the snapshot or persistent state.
Validation
9 passed.2 passedper node.B=4,T=8,C=1152,W=4,state=10):20.387 us -> 18.446 us(1.105x) in the representative run; repeatswere
1.078x-1.104x.all 32 ranks (
bs=[4,32]) and entered generic draft capture withbs=[1,4,32]. 29/32 ranks logged draft-capture completion before anexternal Docker
SIGKILLstopped all four shared containers; the otherthree were interrupted in that same second. Dense3d was enabled and no
KDA shape check, Triton
BSconflict, traceback, or assertion occurred.End-to-end HTTP replay is pending an idle window.
The full run used
SGLANG_RAGGED_VERIFY_MODE=static. Therefore the claimeddense-path benefit comes from dense3d causal-conv, graph-safe padding, and
local draft graph buckets; compact-only ragged scatter/gather is not counted
as an active optimization in this result.
CI States
Latest PR Test (Base): ❌ Run #33762654951
Latest PR Test (Extra): ❌ Run #33762654713
Latest PR Test (AMD ROCm 7.2): ❌ Run #33762654653