From 7d31c79b74f72c0e5297bde4430ba8d2c1d45751 Mon Sep 17 00:00:00 2001 From: agent Date: Fri, 3 Jul 2026 07:11:51 -0700 Subject: [PATCH] [BugFix] Release HiCache prefetch resources on disagg-prefill bootstrap-queue abort ## Motivation When `abort_request()` processes requests still in `disagg_prefill_bootstrap_queue`, it aborts the KV sender but never calls `release_aborted_request()`. Since `_add_request_to_queue()` calls `_prefetch_kvcache()` (which increments `prefetch_tokens_occupied` and creates an `ongoing_prefetch` entry) before adding the request to the bootstrap queue, each aborted bootstrap-queue request permanently leaks `len(prefetch_key)` tokens from the counter. Once the counter exceeds `prefetch_capacity_limit`, `prefetch_rate_limited()` returns `True` for every subsequent request and storage prefetch is permanently disabled for the lifetime of the process. The waiting-queue abort path already calls `release_aborted_request()` (line 3857); the bootstrap-queue path was never updated to match. Introduced in v0.5.12 by #23631 (commit 233048212a). Relates to #26886, #27619. ## Modifications Call `release_aborted_request()` in the bootstrap-queue abort loop, gated on `self.enable_hicache_storage`, matching the waiting-queue pattern. ## Checklist - [x] Format: `pre-commit run --all-files` - [x] The fix is a 3-line addition following the existing pattern --- python/sglang/srt/managers/scheduler.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/python/sglang/srt/managers/scheduler.py b/python/sglang/srt/managers/scheduler.py index 43ddeb397efa..e6205d3fb2d2 100644 --- a/python/sglang/srt/managers/scheduler.py +++ b/python/sglang/srt/managers/scheduler.py @@ -3910,6 +3910,9 @@ def abort_request(self, recv_req: AbortReq): for req in self.disagg_prefill_bootstrap_queue.queue: if recv_req.abort_all or req.rid.startswith(recv_req.rid): logger.debug(f"Abort bootstrap queue request. {req.rid=}") + if self.enable_hicache_storage: + self.tree_cache.release_aborted_request(req.rid) + if hasattr(req.disagg_kv_sender, "abort"): req.disagg_kv_sender.abort()