Skip to content

[HiCache] refactor: retire in-flight storage prefetches through one helper - #41453

Merged
ispobock merged 6 commits into
sgl-project:mainfrom
alphabetc1:refactor-urc-prefetch-teardown
Oct 4, 2026
Merged

ispobock merged 6 commits into
sgl-project:mainfrom
alphabetc1:refactor-urc-prefetch-teardown

Conversation

@alphabetc1

@alphabetc1 alphabetc1 commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Dropping an in-flight storage prefetch was written four times: release_aborted_request, the all-or-nothing discard in _check_hybrid_prefetch_result, revoke_pending_prefetch, and StorageAttachment._release_pending_storage_ops. Each releases the anchor host lock, the buffer-mode prefix context and anchor, the host pages, and the prefetch occupancy, and they had already drifted (two clamp prefetch_tokens_occupied at 0, two do not).

Modifications

  • Add _retire_ongoing_prefetch(request, host_indices, extra_pools) and route all four through it. What each caller releases (the completed prefix, the aux pools, whether pool_transfers_done gates them) is passed in unchanged.
  • _check_hybrid_prefetch_result loses four parameters it no longer needs.
  • The occupancy decrement is clamped at 0 everywhere; this only differs from before when the counter would have gone negative.

Accuracy Tests

  • test/registered/unit/{mem_cache,disaggregation,managers} and the Rust-core run of the shared suite: same results as main. The mock-based storage-cleanup test now wires the new helper like it already wires _prefetch_occupied_span.
  • PD e2e smoke with decode HiCache + file L3 prefetch (Qwen3-8B), flushing L1/L2 between rounds: healthy (96 prefetches). That run mostly covers the success path; the discard paths are covered by the unit tests above.

Checklist


CI States

Latest PR Test (Base): ⏳ Run #37202314137
Latest PR Test (Extra): ❌ Run #37202313945
Latest PR Test (AMD ROCm 10): ❌ Run #37202314347

@github-actions github-actions Bot added hicache Hierarchical Caching for SGLang unified-radix-cache labels Sep 27, 2026
@alphabetc1 alphabetc1 added run-ci CI: run the baseline test suite on this PR run-ci-extra CI: also run the extra suite (requires run-ci) bypass-fail-fast CI: a failing job no longer aborts its siblings (lint still gates) labels Sep 30, 2026
@alphabetc1
alphabetc1 enabled auto-merge (squash) October 4, 2026 16:10
@ispobock
ispobock disabled auto-merge October 4, 2026 16:13
@ispobock
ispobock merged commit 137c084 into sgl-project:main Oct 4, 2026
161 of 185 checks passed
@alphabetc1
alphabetc1 deleted the refactor-urc-prefetch-teardown branch October 4, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bypass-fail-fast CI: a failing job no longer aborts its siblings (lint still gates) hicache Hierarchical Caching for SGLang run-ci CI: run the baseline test suite on this PR run-ci-extra CI: also run the extra suite (requires run-ci) unified-radix-cache

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants