Skip to content

[Fix] hicache: Release late checkpoint claims after cancellation - #5

Merged
ziqifan617 merged 1 commit into
ziqifan617:ziqifan617/kvcr-hicache-storagefrom
PeaBrane:codex/kvcr-confirmation-claim-cleanup
Oct 31, 2026
Merged

ziqifan617 merged 1 commit into
ziqifan617:ziqifan617/kvcr-hicache-storagefrom
PeaBrane:codex/kvcr-confirmation-claim-cleanup

Conversation

@PeaBrane

Copy link
Copy Markdown

Motivation

Cancellation can remove a checkpoint lookup's ownership while confirm_lookup
waits for the native fetch. A successful late result then stores its claim in a
detached dictionary, retaining the checkpoint indefinitely and potentially
accepting a cancelled boundary.

This is a follow-up to sgl-project#45647, based on
its preview head b0506add3eac49bbb2c303e83169ea19c907fb98. It targets the preview
branch so the diff contains only this fix and its regressions.

Modifications

Recheck scope ownership after the wait, release successful late handles when it
has been revoked or replaced, and restrict exception cleanup to the original
scope generation. Add deterministic CPU regression cases for complete/partial
results, sibling and replacement lifetimes, and late-release errors. The tests
exercise the real adapter with the existing transport dependency fixture.

Accuracy Tests

The identical changed adapter methods passed native KVCR/NIXL cancellation,
partial-result, sibling-lifetime, replacement-generation and two-rank checks in
a compatibility runtime. Actual server tests also passed targeted cancellation,
an 18-round cancellation soak, and two mixed cancellation/restart/recovery cycles,
with successful late fetches and no retained claims at quiescence.

Patch application, Python syntax and changed-method AST equivalence pass at the
pinned preview head. The new upstream CPU cases have not been run, and a full
matching upstream runtime is not qualified. This draft needs those tests run on
both the base and fixed adapters. No matching local SGLang environment is installed.

Speed Tests and Profiling

No performance benchmark was run for this change.

Checklist

  • Run upstream formatting/static checks (Ruff, isort and codespell unavailable locally).
  • Add unit regression cases.
  • Execute the new cases and matching upstream suite before review completion.
  • State the compatibility-runtime validation limits.

Signed-off-by: Rudy Pei <yanrpei@gmail.com>
@PeaBrane
PeaBrane marked this pull request as ready for review October 31, 2026 17:06

@ziqifan617 ziqifan617 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Rudy

@ziqifan617
ziqifan617 merged commit 6587556 into ziqifan617:ziqifan617/kvcr-hicache-storage Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants