Repository navigation
perf: add opt-in DRAM descriptor coalescing and UCX batching - #113
Closed
ziqifan617 wants to merge 1 commit into
Closed
ziqifan617 wants to merge 1 commit into
ziqifan617 wants to merge 1 commit into
Conversation
Preserve source/destination byte pairs when grouping remote framework DRAM descriptors. Add a validated optional UCX split-batch setting and native descriptor telemetry, with unit coverage. Defaults remain unchanged. No dependency or model changes.
This was referenced Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add optional transfer controls needed by the SGLang HiCache performance integration, separately from basic enablement and operator documentation. Existing branch/head is preserved at
bb48f3843a292f514becd3efedf29429e272c0af.nixl_ucx_split_batch_size; reject invalid integer/range/backend combinations before native startup.Companion: sgl-project/sglang#45649. The initial HiCache integration sgl-project/sglang#45647 does not require enabling these tuning options.
Scope and tests
Four runtime files and two unit-test files: 285 additions / three deletions overall. Added cases cover split-option validation and backend propagation, coalescing address-pair integrity, overlap/gap behavior and descriptor telemetry.
Focused test targets for a compatible Linux/NIXL environment:
This publication pass checks exact branch identity, diff whitespace and clean composition with current main
0521fb1981ab34fc5c80a8115849c8987e99f76c. It does not claim a fresh Linux/native test execution or an isolated end-to-end speedup for this SDK-only change. The SGLang companion reports a combined-stack matched-workload regression screen; that is not a causal benchmark of this individual PR.Draft / merge gates
The separate SGLang quick-start preview does not depend on these optional tuning settings.
Operator documentation preview (independent): #114