Repository navigation
Linhu/kvcr 45650 land - #6
Merged
ziqifan617 merged 4 commits intoNov 4, 2026
Merged
ziqifan617 merged 4 commits into
ziqifan617 merged 4 commits into
Conversation
…erface Signed-off-by: Moein Khazraee <moein@nvidia.com> (cherry picked from commit 02f0fb9)
A batch drains only after its forward, so the owner kept polling with sleep(0) and competing for the GIL long after the last layer landed. Spin only while transfers are in flight.
Pack the owned rows of all of a layer's buffers into one staging tensor instead of issuing one NCCL all-gather per buffer.
KVCR now converts keys through encode/decode_to/from_framework and encode/decode_to/from_router. Linker keys are already KVCR keys; the router key is the page hash prefix, which spans every pool, so it has no inverse.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates the linker branch behind sgl-project#45650 for KVCR 0.2.0 (KVCR main 5d5fa15, which has sgl-project#133 and sgl-project#161).
Validation: linker unit tests 46 passed against KVCR 5d5fa15; kvcr#129 quick start (head 0e13ac6) run as written on 2x H20 (Qwen3-32B TP1), peer reuse 4224/4268 tokens, remote_deliver/source_write telemetry with no failures, no hint warnings. Requires KVCR >= 5d5fa15.