Repository navigation
[Fix] hicache: Release late checkpoint claims after cancellation - #5
Merged
ziqifan617 merged 1 commit intoOct 31, 2026
Conversation
Signed-off-by: Rudy Pei <yanrpei@gmail.com>
PeaBrane
marked this pull request as ready for review
October 31, 2026 17:06
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Cancellation can remove a checkpoint lookup's ownership while
confirm_lookupwaits for the native fetch. A successful late result then stores its claim in a
detached dictionary, retaining the checkpoint indefinitely and potentially
accepting a cancelled boundary.
This is a follow-up to sgl-project#45647, based on
its preview head
b0506add3eac49bbb2c303e83169ea19c907fb98. It targets the previewbranch so the diff contains only this fix and its regressions.
Modifications
Recheck scope ownership after the wait, release successful late handles when it
has been revoked or replaced, and restrict exception cleanup to the original
scope generation. Add deterministic CPU regression cases for complete/partial
results, sibling and replacement lifetimes, and late-release errors. The tests
exercise the real adapter with the existing transport dependency fixture.
Accuracy Tests
The identical changed adapter methods passed native KVCR/NIXL cancellation,
partial-result, sibling-lifetime, replacement-generation and two-rank checks in
a compatibility runtime. Actual server tests also passed targeted cancellation,
an 18-round cancellation soak, and two mixed cancellation/restart/recovery cycles,
with successful late fetches and no retained claims at quiescence.
Patch application, Python syntax and changed-method AST equivalence pass at the
pinned preview head. The new upstream CPU cases have not been run, and a full
matching upstream runtime is not qualified. This draft needs those tests run on
both the base and fixed adapters. No matching local SGLang environment is installed.
Speed Tests and Profiling
No performance benchmark was run for this change.
Checklist