[HiCache] Replace skip_lock_node_ids with a segment lock protocol by xiezhq-hermann · Pull Request #36848 · sgl-project/sglang

xiezhq-hermann · 2026-08-28T10:36:51Z

Motivation

Lock receipts carried skip_lock_node_ids: a per-component set of node ids
that were tombstones at acquire time, replayed at release so a short-lived
lock could not consume a later load-back or request lock once that tombstone
became a valid device value.

That set had to be built, carried and replayed correctly at every
acquire/release pair, it grew with the tree, and every new call site had to
remember to thread it through. The release decision depended on state captured
somewhere else, and a lost or defaulted receipt silently released another
holder's lock.

Modifications

Replace the set with a segment lock. A lock covers the contiguous node
segment from the locked node up to the boundary stamped by the SWA window walk,
counting every node in it (tombstones included). The receipt alone determines
the release:

IncLockRefResult / DecLockRefParams drop skip_lock_node_ids and carry
the anchor node id, the SWA boundary uuid (None = root) and the
skipped component set. Every release (dec_lock_ref, dec_swa_lock_only,
dec_host_lock_ref) asserts the receipt is replayed on the node its acquire
returned, so a mispaired release fails loud instead of walking another
holder's segment. Req, SessionSlot and DecodeRequest carry the receipt
object itself (req.lock_receipt) instead of scattered fields.
inc_lock_ref(node_id, skip_lock_components=()) replaces the old
skip_lock_components and the interim lock_mamba: the core names no
component, it drives whatever the tree registered through the same
interface, and the decode hold passes (ComponentType.MAMBA,). Meeting
lock_ref == 0 inside a segment is a hard assertion instead of a silent skip.
Every component release refreshes the evictable-leaf sets of the nodes it
unlocks, so a leaf whose last lock is an SWA or Mamba one is readmitted
whatever the release order; the receipt is required at every release seam,
including the host unlock and the component trait.
full_component, swa_component, mamba_component, unified_tree_core and
unified_tree_core_interface move onto the receipt; node splits copy lock
refs and migrate the boundary uuid so an outstanding receipt still releases
exactly (the anchor stays on the deeper half). A Mamba tombstone refilled by
insert while a lock counts it is credited protected, not evictable.
swa_radix_cache, schedule_batch, schedule_policy,
scheduler_components/dynamic_chunk_sizer, disaggregation/decode,
disaggregation/decode_hicache_mixin, session/streaming_session and the
scripted-runtime lock_ref_exhauster are updated at their call sites.
SessionSlot persists swa_prefix_lock_released so closing or aborting a
streaming session after a first-turn early SWA release does not release the
segment twice.
Rust TreeCore (rust/sglang-radix-tree): the same protocol, ported
faithfully, with a ComponentSet bitmask for the skipped set, the anchor
assert, leaf-set refresh in the auxiliary release walks, auxiliaries released
before Full, a required receipt on the host unlock, and a positive-window
assert in the SWA constructor. The PyO3 bindings and
rust_tree_core/adapter.py follow the new interface (the adapter also
rejects a missing or non-positive sliding_window_size up front), so the
shared Python/Rust cache suite runs under both backends.

Review round

The PR is one squashed commit on current main (13469c16d3); the review
round below is folded into it. Addresses the comments from @ispobock and @alphabetc1: anchor on the receipt
(no more trusting the caller's node; the PD hicache restore re-pair now
asserts instead of silently mispairing), no hard-coded Mamba in the generic
core (parametric skip set instead), TestSegmentLockProtocol /
TestSegmentLockFuzz ported to the inspection interface so they run under the
Rust core too, a new TestSegmentLockFuzzWithMamba (FULL+SWA+MAMBA: the
opt-out receipt, the lower-priority cascade on early SWA release, Mamba
evictions), plus the key_len -> value_len and
hicache_restore_lock_receipt renames.

Accuracy Tests

N/A — no change to model forward paths; this is lock bookkeeping.

Speed Tests and Profiling

Neutral: the removed set was allocated, populated and replayed per
acquire/release and scaled with the tree, whereas the receipt is two scalar
fields. Tree micro-benchmarks (test_unified_radix_cache_bench.py --bench,
insert/match/evict/lock_unlock/cache_finished across FULL, FULL+SWA,
FULL+SWA+MAMBA) are within noise of the baseline; lock_unlock is parity or
better in every configuration.

Source commits (internal tree, for provenance)

0497ae074f16
d896398dd0
649c01637e3d

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.

Testing

Ran locally on the rebased head 72ffb39689 (B200 devbox, PyTorch 2.11,
Rust 1.92 per rust-toolchain, SGLANG_ENABLE_ASYNC_ASSERT=true):

Python backend, test_unified_radix_cache_unittest: 1210 passed, 0 failed,
1362 skipped (non-applicable component configs). The lock consumers touched by
the receipt move (test_streaming_session_unit, test_decode_radix_lock_ref,
test_swa_lock_release_lifecycle, test_mamba_donated_alloc_ratio,
test_unified_radix_cache_bench, test_swa_unittest,
test_swa_eviction_boundary, test_scheduler_chunked_req_gate,
test_rust_tree_core, test_rust_extension): 223 passed;
test_rust_tree_core_integration (incl. [Rust TreeCore] Harden runtime and CI parity #37303's stale-handle test with the
receipt fix): 105 passed.
Rust backend, shared suite via test_rust_unified_radix_cache_unittest.py
(production + inspection extensions built from this branch): 2572 ran, OK,
1368 skipped. The segment-lock protocol and fuzz classes now run under the
Rust core; the one case that drives Python component objects directly skips.
cargo test in rust/sglang-radix-tree: 839 passed, 0 failed;
cargo clippy --all-targets -- -D warnings clean in all three CI shapes
(default, python-extension,inspection, --no-default-features --features tch/doc-only); cargo fmt --check clean; ruff check --select F401,F821,UP037,
ruff format --check, isort --check-only and codespell clean on the
changed files.
Red/green on the new regression tests with the respective fix removed: the
Mamba refill credit, the auxiliary-release leaf refresh, and the receipt
anchor mismatch all fail without their fix on both backends.

End-to-end on a real FULL+SWA+MAMBA model (this protocol as landed internally,
thinkingmachines/Inkling-Small-NVFP4, tp=4 on 4x GB300, strict pool checks on):
gsm8k 0.885 / 0.860 on the Python / Rust core (threshold 0.80), bit-exact KL
0.0 on single-turn, prefill cache hit, decode cache hit and HiCache multi-turn
branching on both cores, and a shared-prefix serving stress with HiCache
write_through, ~140 retractions and outputs past the SWA window: 128/128
requests on every pass, identical cached-token counts on both cores, zero lock
or ledger violations.

Original commits

86df8c33ba4968038abdc2a563c9c96a1db3e127

CI States

Latest PR Test (Base): ✅ Run #34329336067
Latest PR Test (Extra): ❌ Run #34329335686
Latest PR Test (AMD ROCm 10): ❌ Run #34329335745

xiezhq-hermann · 2026-08-28T10:40:15Z

@ispobock @hzh0425 can you help take a look? can really use some early feedback on this refactoring.

xiezhq-hermann · 2026-09-02T20:09:50Z

/tag-and-rerun-ci

xiezhq-hermann · 2026-09-03T04:49:46Z

/tag-and-rerun-ci

xiezhq-hermann · 2026-09-03T05:49:02Z

/tag-and-rerun-ci

xiezhq-hermann · 2026-09-03T08:29:03Z

/tag-and-rerun-ci

xiezhq-hermann · 2026-09-03T16:15:02Z

/tag-and-rerun-ci

ispobock · 2026-09-03T16:35:05Z

-    skip_lock_node_ids: dict[ComponentType, set[int]] = dataclasses.field(
-        default_factory=dict
-    )
+    mamba_lock_acquired: bool = False


the release still depends on which node the caller passes to dec_lock_ref, not the receipt alone, so passing the wrong node releases the wrong segment and silently leaks or steals another lock. can we store the anchor node id on the receipt and assert it at release so a mispair cannot happen?

ispobock · 2026-09-03T16:35:05Z


    def inc_lock_ref(
-        self, node_id: NodeId, skip_lock_components: Sequence[ComponentType] = ()
+        self, node_id: NodeId, lock_mamba: bool = True


this hardcodes MAMBA into the generic core (also dec_lock_ref, dec_swa_lock_only), breaking the component polymorphism the core is built on, since the core should drive components through their interface like eviction_priority and never name a concrete type. base kept it parametric via skip_lock_components with the caller passing (ComponentType.MAMBA,), so can we keep it parametric or use a component owned predicate like decode_lock_optional?

ispobock · 2026-09-03T16:35:05Z

+    recomputation (sanity_check) as the per-step oracle."""
+
+    cfg = CacheConfig(
+        components=(ComponentType.FULL, ComponentType.SWA),


the fuzzer is FULL+SWA only, so the hardest paths (lock_mamba=False, the lower priority cascade release, host_uuid migration) never run, and both new suites skipTest on rust. can we add MAMBA here, run the oracle on rust, and validate end to end on a real FULL+SWA+MAMBA model like Inkling?

ispobock · 2026-09-04T04:05:12Z

-            if cd.lock_ref == 0:
-                key_len = len(cur.key)
+            if cd.lock_ref == 0 and cd.value is not None:
+                key_len = len(cd.value)


nit: maybe we can rename key_len to value_len

alphabetc1 · 2026-09-06T17:07:08Z

        self,
        node_id: NodeId,
-        skip_lock_components: Sequence[ComponentType] = (),
+        lock_mamba: bool = True,


Now that inc_lock_ref and dec_lock_ref need to be aware of mamba's logic, doesn't that seem a bit unreasonable?

Suggestion:
skip_lock_components=(ComponentType.MAMBA,)

alphabetc1 · 2026-09-06T17:11:17Z

-                        .insert(node_handle);
-                }
+            let component = Arc::clone(&self.components[i]);
+            if !lock_mamba && component.component_type() == MAMBA {


ditto(https://github.com/sgl-project/sglang/pull/36848/changes#r3944658733)

alphabetc1 · 2026-09-06T17:12:03Z

            }
-            let component = Arc::clone(&self.components[i]);
-            component.release_component_lock(self, node_id, params, /* lock_host = */ false);
+            if !mamba_lock_acquired && ct == MAMBA {


ditto(https://github.com/sgl-project/sglang/pull/36848/changes#r3944658733)

alphabetc1 · 2026-09-06T17:17:03Z

        self,
        node_id: NodeId,
-        params: Optional[DecLockRefParams] = None,
+        params: DecLockRefParams,


Can we store the anchor in the receipt? Right now dec_lock_ref trusts the caller's node_id, so a valid receipt + wrong anchor can release another holder's lock without the > 0 assertions catching it.

alphabetc1 · 2026-09-06T17:31:32Z

+        # boundary, but the prealloc lock and the receipt on the req still
+        # belong to pm.last_device_node; restore the pairing so any release
+        # before the commit hands over the restored lock hits the right node.
+        dr.req.last_node = pm.last_device_node


The need to restore req.last_node here seems to be another symptom that the lock receipt does not fully identify the acquired lock. Could we carry the acquisition anchor in the receipt/lock handle instead? Then rematching the request wouldn't be able to silently break the node/receipt pairing.

alphabetc1 · 2026-09-06T17:39:10Z

    hicache_restored_kv_indices: Optional[torch.Tensor] = None
    hicache_restored_node: Any = None
+    # Receipt for the inc_lock_ref held on hicache_restored_node.
+    hicache_restored_lock_params: Optional[DecLockRefParams] = None


nit: maybe renamed to hicache_restore_lock_receipt?

alphabetc1 · 2026-09-06T17:40:02Z

-        self.tree_cache.dec_lock_ref(prefix_match.last_device_node)
+        req = decode_req.req
+        restored_node = decode_req.hicache_restored_node
+        restored_params = decode_req.hicache_restored_lock_params


nit: maybe renamed to restored_lock_receipt?

xiezhq-hermann · 2026-09-08T00:24:33Z

Rebased onto current main and pushed a third commit addressing the review (thanks @ispobock @alphabetc1):

Anchor on the receipt. IncLockRefResult / DecLockRefParams now carry the node id the lock was taken on, and dec_lock_ref, dec_swa_lock_only and dec_host_lock_ref assert the receipt is replayed on that node. Req, SessionSlot and DecodeRequest carry the receipt object itself (req.lock_receipt), so the PD hicache restore re-pair now asserts on a mispair instead of silently releasing the wrong segment.
No hard-coded Mamba in the core. inc_lock_ref(node_id, skip_lock_components=()) replaces lock_mamba; the receipt records the skipped set and the releases honor it generically (a ComponentSet bitmask on the Rust side). The decode hold passes (ComponentType.MAMBA,).
Fuzz on Rust and with Mamba. TestSegmentLockProtocol / TestSegmentLockFuzz read the tree through the inspection interface and run under the Rust core via test_rust_unified_radix_cache_unittest.py; TestSegmentLockFuzzWithMamba covers FULL+SWA+MAMBA (opt-out receipt, lower-priority cascade on early SWA release, Mamba evictions). Every component release also refreshes the evictable-leaf sets of the nodes it unlocks, and the Rust core releases auxiliaries before Full, so release order no longer decides leaf membership. The host unlock takes a required receipt, and a non-positive SWA window is rejected up front.
Nits: key_len -> value_len, hicache_restore_lock_receipt / restored_lock_receipt.

Validation is in the updated description, including an end-to-end run of this protocol on Inkling-Small-NVFP4 (FULL+SWA+MAMBA, tp=4) on both cores with bit-exact KL and zero lock or ledger violations.

xiezhq-hermann · 2026-09-08T02:29:59Z

CI triage for run 34173204012 (head d90e94f):

base-c-test-8-gpu-h200 (2): test_dsa_glm52_tp_mtp.py::test_bs_1_speed asserts acc_length > 4.0 and measures 3.805 on all six attempts. Upstream main at 20ca564b (this branch's merge base) fails the same shard with the identical 3.805 (run 34169092667, job 101885912193), so this is a pre-existing MTP acceptance regression on main, not something this PR introduces. The tree cache is not on that path for a single bs=1 prompt.
base-b-test-1-gpu-small (5): the shard hit the 30-minute job timeout inside test_expert_pack_mxfp4.py (15/15, est. 7 s, passed in 7 s on the same main run). The 14 files before it, including test_unified_radix_cache_bench.py from this PR, passed with their usual timings. Runner-level hang, unrelated to the lock protocol.
base-b-test-1-gpu-large (2) (the job linked above) completed successfully on retry.

Will re-run the two failed shards once the run finishes.

xiezhq-hermann · 2026-09-08T03:39:22Z

Re-ran the two failed shards (attempt 2): both green.

base-c-test-8-gpu-h200 (2): test_dsa_glm52_tp_mtp.py passed this time with acc_length 4.381 over a 679-token generation, versus 3.805 over a 449-token generation in attempt 1 and in the main run. The bs=1 greedy generation itself differs between runs, so the acceptance length is run-dependent; the 4.0 floor is a flaky gate on both main and this branch.
base-b-test-1-gpu-small (5): test_expert_pack_mxfp4.py completed in 6 s on the re-run (it hung to the 30-minute job timeout in attempt 1).

lluki · 2026-09-09T07:35:16Z

Would it make sense to also refactor host_lock_ref in this PR ? I think then it would also fix the problem in #38480

Lock receipts carried `skip_lock_node_ids`: a per-component set of node ids that were tombstones at acquire time, replayed at release so a short-lived lock could not consume a later load-back or request lock once that tombstone became a valid device value. The set had to be built, carried and replayed correctly at every acquire/release pair, it grew with the tree, and a lost or defaulted receipt silently released another holder's lock. Replace it with a segment lock: a lock covers the contiguous node segment from the locked node up to the boundary stamped by the SWA window walk, counting every node in it (tombstones included). The receipt alone determines the release: - `IncLockRefResult` / `DecLockRefParams` drop `skip_lock_node_ids`; the SWA uuid already on the receipt marks the segment boundary, and the receipt carries the anchor node id. `dec_lock_ref`, `dec_swa_lock_only` and `dec_host_lock_ref` assert the receipt is replayed on the node its acquire returned, so a mispaired release fails loud instead of walking another holder's segment. `Req`, `SessionSlot` and `DecodeRequest` carry the receipt object itself (`req.lock_receipt`). - The core names no component: `inc_lock_ref(node_id, skip_lock_components)` records the skipped set on the receipt (`ComponentSet` bitmask in Rust); the decode hold passes `(MAMBA,)`. Meeting `lock_ref == 0` inside the segment is a hard assertion. - Values materialized under lock (load-back, SWA rebuild, a Mamba tombstone refilled by insert) are credited to protected directly; node splits copy lock refs and migrate the boundary uuid so an outstanding receipt still releases exactly. - Every component release refreshes the evictable-leaf sets of the nodes it unlocks, and the Rust core releases auxiliaries before Full, so a leaf whose last lock is an auxiliary one is readmitted whatever the order. - The receipt is required at every release seam, including the host unlock and the component trait, in Python and Rust. A non-positive SWA window is rejected by the adapter and the Rust constructor. - `SessionSlot` persists `swa_prefix_lock_released` so closing or aborting a streaming session after a first-turn early SWA release does not release the segment twice. - The Rust TreeCore (`rust/sglang-radix-tree`), its PyO3 bindings and `rust_tree_core/adapter.py` are ported to the same protocol. The segment-lock protocol and fuzz tests read the tree through the inspection interface and run under both backends; a FULL+SWA+MAMBA fuzz exercises the Mamba opt-out receipt and the cascade. Co-authored-by: Zhiqiang Xie <zqx@meta.com> Signed-off-by: Zhiqiang Xie <xiezhq.hermann@gmail.com>

xiezhq-hermann · 2026-09-09T08:31:01Z

Rebased onto current main (13469c16d3, 71 commits) and squashed the three commits into one so the Rust conflicts against #37303 were resolved once. New head 72ffb39689.

What the rebase changed (only the six Rust files under rust/sglang-radix-tree differ from the pre-rebase PR; every Python file auto-merged and was re-reviewed hunk-by-hunk against the pre-rebase diff):

[Rust TreeCore] Harden runtime and CI parity #37303 (Result<_, NodeAccessError> everywhere). inc_lock_ref / dec_lock_ref / dec_swa_lock_only / inc_host_lock_ref / dec_host_lock_ref keep the receipt protocol (required params, ComponentSet skip set, anchor assert) and now return Result<_, NodeAccessError> like the rest of the core; the PyO3 bindings map it with node_access_error, so a stale handle raises KeyError without poisoning the mutex. assert_receipt_anchor_ compares handles directly since try_resolve is gone.
Tests. Both parents' test changes are kept: the PR's segment-protocol tests plus [Rust TreeCore] Harden runtime and CI parity #37303's stale-handle / shortened-prefetch tests, adapted to the Result signatures. test_stale_handle_operations_raise_key_error_without_poisoning_the_core (new in [Rust TreeCore] Harden runtime and CI parity #37303) called the three release ops without a receipt, which the protocol now rejects with TypeError before the stale handle is seen; it passes DecLockRefParams() so the stale handle reaches the core and raises KeyError as the test intends.
Dropped a # noqa: B036 tag that ruff 0.15.1 flags as an unknown rule.

Local validation on the new head (B200 box, SGLANG_ENABLE_ASYNC_ASSERT=true):

check	result
`cargo test` (rlib shape)	839 passed, 0 failed, 1 ignored
`cargo clippy --all-targets -D warnings` (default / `python-extension,inspection` / `--no-default-features --features tch/doc-only`) + `cargo fmt --check`	clean
`test_unified_radix_cache_unittest.py` (Python core)	1210 passed, 0 failed
`test_rust_unified_radix_cache_unittest.py` (Rust core, shared suite)	2572 run, OK
`test_rust_tree_core_integration.py`	105 passed
the other PR-touched test files (`test_decode_radix_lock_ref`, `test_streaming_session_unit`, `test_swa_lock_release_lifecycle`, `test_swa_eviction_boundary`, `test_swa_unittest`, `test_mamba_donated_alloc_ratio`, `test_rust_tree_core`, `test_unified_radix_cache_bench`, `test_scheduler_chunked_req_gate`, `test_rust_extension`)	223 passed
ruff (F401,F821,UP037) / ruff format / isort / codespell / check-no-bare-pytest-main on the changed files	clean

xiezhq-hermann · 2026-09-09T20:44:57Z

CI on 72ffb39689: the main PR workflow (run 34329336067) is green (60 jobs, pr-test-finish success). Attempt 1 lost base-b-test-1-gpu-small (5) to the test_openai_completion_rust logprob-parity flake (that server runs with --disable-radix-cache), attempt 2 passed it and then hit the test_expert_pack_mxfp4.py JIT-load hang (#38408); attempt 3 re-ran the shard clean.

The remaining red checks are platform workflows, each reproduced on unrelated PRs today and none on a path this PR touches:

XPU: pip cannot resolve sgl-kernel (unavailable) during install.
MLX: mlx/test_scheduler_mixin.py::test_loop_exits_after_shutdown_req raises _StopLoop through ingest_requests() ([Scheduler] Unify per-iteration request intake into ingest_requests() #38389); same on other PRs and a main-based run.
ROCm: test_umbp_store ('>' not supported between 'MagicMock' and 'int', 6+ other PRs); test_deterministic::test_prefix_with_logprobs logprob drift on MI300 Triton (4 other branches); AttributeError: 'Tensor' object has no attribute 'format_ue8m0' in deepseek_weight_loader.py on all four MI300 8-GPU shards (DeepSeek-V3/V3.2, Kimi-K2, MoRI EP); on MI35x the DSV4-Flash-FP8 SGLANG_OPT_FP8_WO_A_GEMM dtype check, a NIXL NIXL_ERR_BACKEND, and the quark mxfp4 gsm8k gate (0.80 vs 0.85).
NPU: perf-16 Qwen3.5-397B TPOT ~100 ms vs the 50 ms gate, reproduced on rerun; four other PRs fail it identically today, and the test runs with --disable-radix-cache (ChunkCache). The previous head of this PR passed it at 42 ms two days ago on the old base.
Extra / Extra (AMD): fail at the run-ci-extra label gate only (not opted in).

xiezhq-hermann · 2026-09-09T21:55:30Z

Would it make sense to also refactor host_lock_ref in this PR ? I think then it would also fix the problem in #38480

I think it's worth doing it but probably not in this PR, which has been fairly large already.

* origin/main: (27 commits) [Simulator] Give the OFFLINE/BLOCKING comparison tolerances real headroom (sgl-project#38732) [Config] msgspec.Struct for the config tier (sgl-project#38753) [AMD] ci: move the miles nightlies from rocm700 to rocm10 (sgl-project#37495) [Config] One writer for the declaration stash; no exception to the write seal (sgl-project#38752) docker(xpu): drop redundant setvars.sh from torch_memory_saver RUN (sgl-project#38665) [XPU][Fix] Pack device-pointer tables as uint64 to avoid 64-bit address overflow (sgl-project#35051) [CI] Temporarily disable GB300 tests (sgl-project#38770) [diffusion] feat: spill large tensors over shared memory like numpy arrays (sgl-project#38656) [diffusion] refactor: refactor utility ownership and document helper placement (sgl-project#38699) [NPU]Support GLM5.2 and FP8 DSA&Indexer kvcache for 950 (sgl-project#38250) [CI] Answer unrecognized slash commands instead of skipping silently (sgl-project#38736) [AMD] Parallelize aiter spec-decode KV index building over token blocks (sgl-project#37659) [DSv4] Integrate TRT-LLM DSv4 Attention for SM100/103 (sgl-project#30805) Add Opt-In for GLM-5.3 Flash breakable prefill CUDA graphs (sgl-project#38522) [CI] Install helion 1.4.0 for the KDA Helion kernel tests (sgl-project#38688) [Rust] Gate health on startup warmup completion (sgl-project#37994) [HiCache] Replace skip_lock_node_ids with a segment lock protocol (sgl-project#36848) feat: add optimized Domino rollout to DFlash V2 (sgl-project#36899) [CI] Add /run-full-ci and /run-extra-ci slash commands (sgl-project#38734) [Model] Support GLM-5.3 Flash NVFP4 loading (sgl-project#38621) ...

Conflict in schedule_batch.py: main's sgl-project#36848 replaced Req.skip_lock_node_ids (and swa_uuid_for_lock) with a single lock_receipt: DecLockRefParams. Took main's removal and kept this branch's kv_rotation_base field beside it. Ported test_page_interleave_shard.py's _GraftReq stub to the same protocol (it set the two removed attrs; auto-merge left it stale since the file is new on this branch). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…l-project#36848)

github-actions Bot added documentation Improvements or additions to documentation hicache Hierarchical Caching for SGLang unified-radix-cache labels Aug 28, 2026

xiezhq-hermann assigned hzh0425 and ispobock Aug 28, 2026

xiezhq-hermann assigned alphabetc1 Aug 28, 2026

ispobock mentioned this pull request Sep 1, 2026

[HiCache] Preserve restored SWA lock ownership #36350

Closed

4 tasks

xiezhq-hermann force-pushed the hicache-segment-lock-protocol branch from 39de364 to 0c94f3e Compare September 2, 2026 20:09

github-actions Bot added the run-ci label Sep 2, 2026

xiezhq-hermann marked this pull request as ready for review September 2, 2026 20:49

xiezhq-hermann requested review from ByronHsu, Duyi-Wang, HaiShaw, ShangmingCai, Ying1123, alphabetc1, hanming-lu, hnyls2002, huangtingwei9988, hzh0425, ispobock, merrymercy, sogalin and yizhang2077 as code owners September 2, 2026 20:49

xiezhq-hermann assigned Jialin Sep 2, 2026

xiezhq-hermann requested a review from Jialin September 2, 2026 20:50

xiezhq-hermann added the bypass-fastfail label Sep 2, 2026

xiezhq-hermann force-pushed the hicache-segment-lock-protocol branch from 0c94f3e to b4ea300 Compare September 3, 2026 04:49

xiezhq-hermann force-pushed the hicache-segment-lock-protocol branch from b4ea300 to b9177d4 Compare September 3, 2026 05:49

sglang-bot mentioned this pull request Sep 3, 2026

CUDA Coredump Tracker #26340

Open

xiezhq-hermann force-pushed the hicache-segment-lock-protocol branch from b9177d4 to ec9e36c Compare September 3, 2026 08:29

ispobock reviewed Sep 3, 2026

View reviewed changes

bingxche mentioned this pull request Sep 3, 2026

[CI Monitor] Daily Report - 2026-09-03 bingxche/sglang-ci-bot#184

Open

ispobock reviewed Sep 4, 2026

View reviewed changes

alphabetc1 reviewed Sep 6, 2026

View reviewed changes

xiezhq-hermann force-pushed the hicache-segment-lock-protocol branch from f50f7bc to d90e94f Compare September 8, 2026 00:24

Jialin mentioned this pull request Sep 8, 2026

[HiCache]: Fix host lock ownership across radix splits #38480

Open

5 tasks

xiezhq-hermann force-pushed the hicache-segment-lock-protocol branch from d90e94f to 72ffb39 Compare September 9, 2026 08:28

xiezhq-hermann merged commit beaf3d9 into main Sep 9, 2026
309 of 339 checks passed

xiezhq-hermann deleted the hicache-segment-lock-protocol branch September 9, 2026 21:55

alphabetc1 mentioned this pull request Sep 10, 2026

[HiCache] fix: preserve SWA host lock boundaries across splits #38835

Merged

mqhc2020 pushed a commit to mqhc2020/sglang that referenced this pull request Sep 15, 2026

[HiCache] Replace skip_lock_node_ids with a segment lock protocol (sg…

87015f6

…l-project#36848)

Uh oh!

Conversation

xiezhq-hermann commented Aug 28, 2026 • edited by merrymercy Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Review round

Accuracy Tests

Speed Tests and Profiling

Source commits (internal tree, for provenance)

Checklist

Testing

Original commits

CI States

Uh oh!

xiezhq-hermann commented Aug 28, 2026

Uh oh!

xiezhq-hermann commented Sep 2, 2026

Uh oh!

xiezhq-hermann commented Sep 3, 2026

Uh oh!

xiezhq-hermann commented Sep 3, 2026

Uh oh!

xiezhq-hermann commented Sep 3, 2026

Uh oh!

xiezhq-hermann commented Sep 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xiezhq-hermann commented Sep 8, 2026

Uh oh!

xiezhq-hermann commented Sep 8, 2026

Uh oh!

xiezhq-hermann commented Sep 8, 2026

Uh oh!

lluki commented Sep 9, 2026

Uh oh!

xiezhq-hermann commented Sep 9, 2026

Uh oh!

xiezhq-hermann commented Sep 9, 2026

Uh oh!

xiezhq-hermann commented Sep 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

xiezhq-hermann commented Aug 28, 2026 •

edited by merrymercy

Loading