* fix(detokenizer_manager.py): fix truncated decoded output by Titan-p · Pull Request #581 · sgl-project/sglang

Titan-p · 2024-07-02T09:50:13Z

ignore � character

…cter '�'

Titan-p · 2024-07-02T09:51:15Z

hnyls2002 · 2024-07-03T23:10:06Z

+                if new_text.endswith('�'):
+                    new_text = new_text[:-1]


Have you run this code?

UnboundLocalError: local variable 'new_text' referenced before assignment

…ach arm prove its own validity The 2026-08-05 window produced no barlink verdict because the load ran in a regime the crash never occupied. This rebuilds the load profile around the variable that actually saturated and adds the instrumentation that would have caught the mistake in the first minute instead of after two arms. What consumes mamba state is the number of DISTINCT cached prefixes -- the mamba radix cache holds one state per cached prefix under mamba_radix_cache_strategy='extra_buffer'. The old generator gave each session its own forever-growing conversation, so distinct prefixes grew without bound. Sessions now cycle a FIXED pool of six long prefixes, which pins the mamba footprint flat over the soak while each request still carries a large cached-token count plus a fresh 600-2000 token chunk -- the crash boot's shape. The band was then measured rather than assumed, from the two logs: crash boot 5 mamba median 0.25 max 0.80 running median 3.0 max 4 rejected arm 2 mamba median 0.18 max 1.00 running median 3.0 max 4 Two corrections follow from that, and both matter: * The crash boot PEAKED AT 0.80. A safety valve at the band ceiling would have throttled exactly the excursions the crash exhibited, and would have flagged the crash boot itself as out of regime. The valve therefore sits at near-saturation (0.88), not at the band edge. * The rejected arm's MEDIAN was lower than the crash boot's. Median is not what distinguished them -- saturation is. Validity is judged on the median band (0.20-0.35), failure on saturation (>= 0.95). Concurrency was never the problem: both runs sat at running-req median 3.0, max 4, which already matched the crash. Each arm now reports its own validity. A sampler reads the server's own log lines and the run prints MAMBA BAND / REGIME / RUNNING-REQ, so a barlink verdict from an out-of-regime arm is visibly not trustworthy rather than silently wrong. Can-fail proof, replaying the verdict logic over the two real logs: the crash boot scores REGIME OK (median 0.25, peak 0.80) and the rejected arm scores REGIME FAIL (saturated 1.00). Adds the `soak` arm (20 min) for the dual-acceptance window: sgl-project#583's abort path under the crash regime, and sgl-project#581's mamba pool floor live proof in the same run. The verdict block now reports both. sgl-project#581's half is negative evidence -- neither "Not enough space for mamba ping pong idx" nor "Not enough space for mamba cache" may reappear -- plus the regime report showing the run was actually loaded while staying clear. Pin-budget lines are counted and their ABSENCE is reported explicitly, so a soak that logged nothing about pins is recorded as leaving pin behaviour unproven rather than passing by silence. Not yet merged: fix/mamba-pool-floor-581 is still uncommitted in /spinning/wt-mamba-floor (8 modified files, 2 new, 0 commits), so there was nothing to take. The soak needs it -- without the floor the arms die on the asserts again. Validation, no GPU: bash -n, embedded-Python AST parse, codespell clean, and the regime instrument's can-fail proof above.

…ry markers verbatim and guard them with a test The soak reads both acceptances out of the server log by matching fixed strings, so the strings are now pinned to the sources that must contain them rather than matched by a loose pattern. Markers taken from ee8d1b5: hi_mamba_radix_cache.py "mamba write-through pin budget reached" memory_pool.py "deferring this batch" (+ mamba_evictable= / mamba_protected= starvation fields) mamba_component.py "skipping this cache insert" mamba_pool_floor.py "pinned checkpoint" (the floor derivation line) Writing the pins exposed one of my own: "deferring this batch of" is NOT contiguous in the source -- memory_pool.py wraps the format string between "batch " and "of %d request(s)". It matches the formatted log line but cannot be verified against the source, which is precisely how a pin rots unnoticed. The harness and the test now both use the shorter contiguous fragment, and the reason is written down at both sites. test_soak_harness_pins_583.py guards this in three directions: * every pinned literal must exist verbatim in its source file; * every `grep -F` in the harness must be a REGISTERED pin -- this is the direction that actually rots, where someone adds a marker, never registers it, and it quietly stops matching; * the two death asserts must still EXIST in memory_pool.py. The soak proves their absence from a run's log, and you cannot prove the absence of a string that no longer exists anywhere -- deleting them would make the acceptance pass vacuously. That third check also settled how to read the sgl-project#581 verdict. Both asserts are still live `assert` statements (memory_pool.py:1476-1482); sgl-project#581 makes them UNREACHABLE via the floor, the admission gate and graceful back-off elsewhere, rather than converting them into warnings. So their appearance in a log remains a hard failure, and the harness is right to count them that way -- while "deferring this batch" / "skipping this cache insert" are the GRACEFUL paths and are reported as back-off, not death. Pin-budget acceptance is reported honestly in both directions: observing the budget engage is the proof, and silence is printed as "pin behaviour remains UNPROVEN by this arm" rather than folded into a pass. Soak config verified against the new parse-time floor refusal: --max-mamba-cache-size 96 with --max-running-requests 4 gives a hard floor of 20 (4 x (1 active + 2 ping-pong + 1 donation + 1 pinned checkpoint)), so the soak boots with 76 slots of headroom and does not trip the refusal. Tests (no GPU): test_mamba_pool_floor.py + test_barlink_device_abort_583.py 38 passed on the merged tree; test_soak_harness_pins_583.py 4 passed / 9 subtests. ruff and codespell clean. A full test/registered/unit/mem_cache sweep is running; test_hicache_nixl_storage.py fails to collect there for a missing optional 'nixl' module, which is environmental and untouched by sgl-project#581.

…gl-project#581) Operator review raised a sibling of defect (1): _req_inc_lock_ref (schedule_policy.py:1004-1007) discards IncLockRefResult.skip_lock_node_ids for REQUEST locks, and the paired release -- dec_lock_ref(req.last_node) in cache_unfinished_req (mamba_radix_cache.py:886) and cache_finished_req (:683, :731) -- passes no params. That lock would steal a mamba ref exactly like the admission lock did, IF req.last_node could be a mamba tombstone when it is taken. REFUTED for the running path, but only by an emergent invariant spanning two files, so it is now asserted mechanically instead of argued: match_prefix DOES hand out mamba tombstones as last_device_node (hi_mamba_radix_cache.py:1196,1214 select best_last_node on "mamba_value is not None OR mamba_backuped"; :1261-1264 walks up only over KV-evicted nodes), and init_next_round_input assigns it straight to req.last_node (schedule_batch.py:1290-1297). What saves the request lock: mamba_backuped => backuped a mamba host copy is only ever written where the KV host copy is written too (mamba_backup_commit at :2278, called from write_backup right after node.host_value is set; _insert_helper_host at :2182, likewise) => last_host_node (:1266-1268) stops AT the tombstone => mamba_host_hit_length == 1 (:1270-1272) => Req.needs_host_load_back() is True (schedule_batch.py:1124-1130) => init_load_back runs inside _lock_node (schedule_policy.py:1324-1331) and resolves the tombstone before _req_inc_lock_ref (:1359,1367,1412). The other two exits close too: init_load_back's fallback walk (hi_mamba_radix_cache.py:508-511) skips mamba_evicted nodes, and a node selected as best_last_node with mamba_value None is necessarily mamba_backuped, so load_back always puts it in mamba_restore_nodes (:411-412). TestAdmissionLockPrecondition asserts each link on the shape where the KV side gives NO signal at all: an INTERNAL tombstone that keeps its device KV, so host_hit_length is 0 and mamba_host_hit_length alone forces the load back. If a future path ever creates a mamba host copy without a KV host copy, these go red -- and the fix is then to thread the skip set through Req (like swa_uuid_for_lock) to the release sites. Tests: test_mamba_lock_ref_pairing_581.py 9 passed planting the invariant break (host_value = None on a mamba-backuped tombstone) turns all 3 new tests RED, the last one with exactly the sibling symptom: "unexpectedly None : the request lock would be taken on a tombstone and its paramless release would steal a ref" test_alloc_req_slots_names_the_pool_583.py test_mamba_slot_starvation.py test_mamba_pool_floor.py test_mamba_unittest.py test_decode_radix_lock_ref.py test_mamba_lock_ref_pairing_581.py 8 failed, 58 passed -- failure set IDENTICAL to the pre-change baseline (8 pre-existing "No accelerator" failures in test_mamba_unittest.py) hermetic randomized workload, seeds 0-9: match_prefix handed out a mamba tombstone as last_device_node 1354 times; in ZERO of those did the tombstone survive to the _req_inc_lock_ref equivalent Not covered here: the PD-decode path assigns req.last_node = hicache_restored_node (decode_hicache_mixin.py:309) with its own inc_lock_ref(:232) and paramless releases (:178, :297, on a DIFFERENT node), under --disaggregation-mode decode. Out of the running config; flagged as a separate audit.

…drain (sgl-project#581) Boot 20 on 3b1c6d9 (which carries 2915f6d + 1d379c5) died with the unchanged signature: one conversation, ~350 short prefills over an ~81k-token cached prefix, mamba num climbing exactly +1 per turn (92,93,95,96) with #running-req 1, then "mamba_evictable=0 mamba_protected=93" and the alloc_req_slots raise. The first two defects were real but not sufficient. Root cause of the surviving ramp: the ack drain is gated on a cross-rank MIN over a RANK-LOCAL quantity. writing_check / loading_check each count "how many of MY acks are ready", all_reduce(MIN) that count across the TP group, and drain the reduced number. But the transfer queues are rank-local by construction: write_backup backs a node up on one rank and skips it on another (host pool full at :417-421, pin budget at :376-393, parent not backed up at :360-363), and under uneven TP/DCP the ranks' host pools differ by design -- scheduler.py says so at the prefetch site: "RANK-LOCAL: `backuped` means full KV present in THIS rank's host pool ... it can be true here and false on a peer for the same node". So a rank with an EMPTY queue contributes 0, and the MIN then freezes the drain on EVERY rank. On the ranks that do transfer, every pin -- each of which makes a mamba checkpoint unevictable -- is held forever: protected ratchets by one per cached checkpoint, evictable falls to zero, and only the transferring rank hits the wall. Exactly the observed shape. The write side had a second, self-inflicted variant: the scan was gated on `len(self.ongoing_write_through) > 0`. Upstream HiRadixCache (:1024) and UnifiedRadixCache (:2790) gate only on the CONSTANT `pp_rank`, and HiRadixCache's comment names the hazard verbatim ("ongoing_write_through can diverge across ranks (e.g. write_backup returning 0 on a subset under host memory pressure)"). With the state gate, a rank holding acks for nodes that had left the tree -- `_forget_write_through` pops the registry entry, the ack still arrives -- reported "0 ready" forever and stalled the whole group. Which pin carried the production ramp: the log shows NO "pin budget reached" warning although protected reached 93 of 96, so the write-through pins stayed under their cap (96 - hard_floor(4 running x 5) = 76). Load-back pins have no budget at all, so the unbudgeted load side is what ran to 93. Both sides are fixed and both have a falsifier. Fix: `_count_ready_acks` is now the single drain-budget helper for both queues. Every rank still enters exactly one all_reduce per check (NCCL op sequence unchanged, no TP > 1 deadlock), but a rank with an EMPTY queue contributes a no-constraint sentinel instead of 0, and each rank pops at most what it actually has ready. Ranks that all have queues still throttle to the slowest live transfer, so they never run further apart than before. Tests (hermetic, CPU only; two threads simulate the TP ranks with a MIN all_reduce over a barrier): test_mamba_lock_ref_pairing_581.py 16 passed reverting ONLY hi_mamba_radix_cache.py -> 4 failed, on the symptom: test_a_rank_with_no_backups_does_not_stall_the_drain AssertionError: 12 != 0 : write-through pins were never released test_a_rank_with_no_loads_does_not_stall_the_load_back_drain AssertionError: {7: <TreeNode object...>} != {} (pins never drained) test_pins_are_bounded_by_in_flight_copies_not_by_request_count AssertionError: 12 != 0 test_stale_ack_without_registry_entry_does_not_stall_the_drain AssertionError: Lists differ (ack queue never emptied) test_ranks_that_both_back_up_stay_in_lockstep and test_single_rank_drain_needs_no_collective guard the two directions the fix must NOT change. TestMultiTurnRetireReturnsEveryCheckpoint is a NEGATIVE result, labelled as such: the production shape on ONE rank passes with and without the fix. The ramp needs TP > 1, which is why every single-rank probe run so far came back clean. test_alloc_req_slots_names_the_pool_583.py test_mamba_slot_starvation.py test_mamba_pool_floor.py test_mamba_unittest.py test_decode_radix_lock_ref.py test_mamba_lock_ref_pairing_581.py 8 failed, 65 passed -- failure set IDENTICAL to the pre-change baseline (8 pre-existing "No accelerator" failures in test_mamba_unittest.py) test/registered/unit/distributed 3 failed, 2377 passed, 12 skipped, 726 subtests passed -- the same 3 pre-existing test_dcp_context_ceiling.py failures as before the change the randomized single-rank workloads from the earlier commits still end protected=0 on every seed (they cannot exercise this defect: tp_world_size 1) ruff and isort clean on both files.

…#581 Boot 24 (tree 775c809, carrying all three previous commits) died with the unchanged signature, and the crash log only reports `mamba_protected` in the dying breath. This is the field diagnostic that shows the ramp WHILE it climbs, so the next boot attributes it to a queue depth and a call site instead of another round of inference. `SGLANG_MAMBA_PIN_TRACE=N` emits one line per rank every N scheduler ticks from `check_hicache_events` (the per-tick hook, scheduler.py:3847): MAMBA-PIN-TRACE tick=2 ack_write=0 ack_load=0 wt_pins=0 wt_inflight=0 lb_pins=0 ongoing_wt=0 ongoing_lb=0 protected=0 evictable=3 mamba_avail=13 ops[dec@_drain_acked_write=3 dec@load_back=1 dec@loading_check=1 dec_mamba@_drain_acked_write=3 dec_mamba@loading_check=1 inc@load_back=2 inc@write_backup=3 inc_mamba@load_back=1 inc_mamba@write_backup=3] ack_write/ack_load transfer queue depths (a FREEZE shows as growth here) wt_pins/lb_pins outstanding pins, summed over the counting registries wt_inflight queued D->H copies (pins <= inflight by construction) ongoing_wt/ongoing_lb registry sizes protected/evictable mamba SLOT counts (not ref counts -- a node with two refs is one slot) mamba_avail free slots in the state pool ops[...] inc/dec_lock_ref traffic since the PREVIOUS line, attributed to the immediate caller. `inc`/`dec` count calls, `inc_mamba`/`dec_mamba` count the ones where a MAMBA ref actually changed hands -- the only ones that can exhaust the pool. This is the split the coordinator asked for: cache_unfinished_req vs write_backup vs load_back name themselves. Reading it: a RATE mismatch shows as inc_mamba@X consistently exceeding dec_mamba@* with the ack queues near zero; a FREEZE shows as ack_write or ack_load growing without bound; a true ref leak shows as protected climbing while both queues stay empty and the per-site sums balance. Default (unset / 0) is off: `_emit_pin_trace` is never called and each lock operation costs one attribute test. Only HiMambaRadixCache is traced -- it is the class production runs. Suggested deployment: N=50 rather than 1. Decode ticks are fast, and one line per tick per rank would itself become the load. Tests: test_mamba_lock_ref_pairing_581.py 20 passed TestPinTrace covers: the line renders with every field; the counters reset between lines; N throttles the emission (6 ticks at N=3 -> 2 lines); and with the env unset the traced path is never entered. test_alloc_req_slots_names_the_pool_583.py test_mamba_slot_starvation.py test_mamba_pool_floor.py test_mamba_unittest.py test_decode_radix_lock_ref.py test_mamba_lock_ref_pairing_581.py 8 failed, 69 passed -- failure set IDENTICAL to the pre-change baseline (8 pre-existing "No accelerator" failures in test_mamba_unittest.py) ruff and isort clean on all three files.

…uns (sgl-project#581) Boot 26 emitted ZERO trace lines with SGLANG_MAMBA_PIN_TRACE=50 armed: Tree cache initialized: source=default impl=UnifiedRadixCache hybrid_swa=False hybrid_ssm=True hierarchical=True streaming_wrapped=False The first trace landed in HiMambaRadixCache, which this configuration never constructs. Same env flag, same one-line-per-N-ticks shape, same per-site ops split, now on the live class: MAMBA-PIN-TRACE impl=unified tick=2 ack_write=? ack_load=? wt_mamba_pins=0 lb_mamba_pins=0 ongoing_wt=0 ongoing_lb=0 ongoing_backup=0 protected=0 evictable=2 mamba_avail=18 ops[dec@<module>=2 dec_mamba@<module>=1 inc@<module>=2 inc_mamba@<module>=1] Wiring: * `UnifiedRadixCache._emit_pin_trace` runs at the end of its own `check_hicache_events` (after writing_check/loading_check), the per-tick hook the scheduler already calls. * `inc_lock_ref` / `dec_lock_ref` resolve the calling site ONCE per lock call (`_pin_trace_begin`) instead of once per component, and the MAMBA component tags its own accounting with that site from `acquire_component_lock` / `release_component_lock` (unified_cache_components/mamba_component.py). `inc`/`dec` count calls; `inc_mamba`/`dec_mamba` count the ones where a MAMBA ref actually moved -- the only ones that can exhaust the state pool. Host-side mamba locks are tagged `*_mamba_host` so they cannot be confused with device pins. * `wt_mamba_pins` / `lb_mamba_pins` resolve through the unified registries: `_OngoingWriteThrough` / `_OngoingLoadBack` carry the acquire's `lock_params`, so an entry whose acquire SKIPPED mamba (tombstone at acquire time) is correctly not counted as holding a mamba pin. * `ack_write` / `ack_load` print "?" when no cache controller is attached (non-hierarchical builds), so the line is safe on every configuration. Default (unset / 0) is off: `_emit_pin_trace` is never called and each lock call costs one attribute test. Deploy with N=50 rather than 1. WHY UNIFIED, AND IS HiMambaRadixCache REACHABLE AT ALL `registry.py:106-109` (`default_radix_cache_factory`): if ctx.enable_hierarchical_cache: if ctx.is_hybrid_ssm or ctx.is_hybrid_swa: # HybridModel launches HiCache via UnifiedRadixCache by default. return _create_unified_radix_cache(ctx, server_args, params) `--enable-hierarchical-cache` plus a hybrid-SSM model therefore ALWAYS yields UnifiedRadixCache with ComponentType.MAMBA added to tree_components (registry.py:165-195); `source=default` in the boot line means no `--radix-cache-backend` override was in play. `MambaRadixCache` is reachable only at registry.py:128-131, i.e. only when hierarchical cache is OFF. `HiMambaRadixCache` has NO construction site anywhere in the tree (verified by grep for `HiMambaRadixCache(`): it is unreachable on every configuration, not merely on this one. The three earlier commits on this branch are therefore latent-path hardening -- correct, hermetically proven, and byte-irrelevant to production until something constructs those classes. Tests: test_unified_mamba_pin_trace_581.py 7 passed line renders with every field; release attributed to its own site; a tombstone lock counts as `inc` but NOT as `inc_mamba`; counters reset between lines; N throttles emission (6 ticks at N=3 -> 2 lines); default off; and `_mamba_pins_in` ignores registry entries whose acquire skipped MAMBA. test_unified_mamba_pin_trace_581.py test_unified_radix_hicache_dispatch.py test_mamba_lock_ref_pairing_581.py 40 passed test_unified_mamba_views.py 8 skipped (needs an accelerator; no signal on CPU either way) full unified suites, port applied vs port reverted (same run, self-restoring harness): AFTER 851 failed, 29 passed, 656 skipped BASE 851 failed, 29 passed, 656 skipped and the failure SETS diff identical. The 851 are the pre-existing accelerator-required failures in test_unified_radix_cache_unittest.py; the port adds none. The shared unified fixture builds on `get_device()`, so the new tests pin it to CPU to stay hermetic. ruff, black and isort clean on all three files.

…RadixCache (sgl-project#581) This is the sgl-project#581 exhaustion itself, on the class production actually runs. The instrumented boot resolved it: on every ramping line, both ramping ranks, ack_write == wt_mamba_pins == ongoing_wt == protected climbed together (TP0 10 -> 71 over ~4 min; lb_mamba_pins=0, ack_load=0 throughout), and when the load stopped the queues did NOT drain -- TP0 frozen at 71 across thousands of idle ticks. So the ratchet is 100% write-through pins whose acks sit undrained, and a write-through pin also makes a mamba checkpoint unevictable: protected rises one per cached checkpoint until the state pool is gone. MECHANISM (unified_radix_cache.py:2792-2807, pre-fix): `writing_check` counts the ready finish_events of its RANK-LOCAL `cc.ack_write_queue`, all_reduces that count with ReduceOp.MIN, and pops the reduced number. A rank with an EMPTY queue contributes 0, so the MIN conflates "nothing to drain" with "not finished yet" and pins the drain at 0 on EVERY rank. The function's own comment (:2792-2793) already named the divergence -- "ongoing_write_through can diverge across ranks (e.g. write_backup returning 0 on a subset)" -- without drawing the consequence for the reduction. `loading_check` (:2817-2836) had the identical shape over `ack_load_queue`. The trace's rare single drains are the moments the idle rank happened to hold exactly one ready ack, so the MIN came out 1. This is the same defect as 8bca9d3, which fixed it in `hi_mamba_radix_cache.py` -- a class with NO construction site anywhere in the tree. The sentinel semantics are ported here, to the live class, with the clamp property kept: a rank with an empty queue contributes a no-constraint sentinel instead of 0, and each rank pops at most what it actually has ready (the reduction can now come back as the sentinel). Ranks that all hold queues still throttle to the slowest live transfer. Every rank still enters exactly one all_reduce per check, so the NCCL op sequence is unchanged; the PP path keeps its `pp_rank == 0` scan and clamps by its own queue length so a propagated sentinel can never overrun. The `write_back=True` blocking path (:2780-2790) is rank-local and untouched. Trace hooks intact. WHY THE RANKS DIVERGE (what arms the freeze). `write_backup` gives up on purely RANK-LOCAL conditions: * :1786-1793 the rank's host pool cannot fit the node and `evict_host` cannot free enough -> return 0; * :1800-1801 `cache_controller.write` returns None (host allocation failed); * :1765-1769 the recursive "parent must be backed up first" invariant -- once it fails on a rank, every descendant fails too, so the rank LATCHES into backing nothing up. The host pool is `hicache_ratio` x the rank's own device KV pool, and under uneven TP (--rank-tp-ratio, 7/3/3-class here) those differ by construction. The ack queues therefore differ in LENGTH, not just in completion timing -- which is precisely the case the MIN mishandles. The trace's shape matches: TP2 never took a write-backup pin at all (no inc@write_backup in its ops[], ack_write=0 throughout) and TP1 ramped to 31 and then PLATEAUED, the latch signature of :1765-1769. Code-certain: the three rank-local give-up sites. Inferred from the trace, not proven here: that pool SIZING is what put TP2 at zero specifically. Tests (hermetic, CPU only; three threads simulate the TP ranks with a barrier MIN all_reduce): test_unified_mamba_pin_trace_581.py 11 passed reverting ONLY unified_radix_cache.py -> 3 failed, on the symptom: test_an_idle_rank_does_not_freeze_the_write_drain AssertionError: 0 != 4 : backing-up rank never drained test_an_idle_rank_does_not_freeze_the_load_drain AssertionError: 0 != 4 test_queues_empty_when_no_new_backups_arrive queues still full after idle rounds (the live 71/31/0 freeze) test_ranks_with_queues_stay_throttled_to_the_slowest passes both ways: it guards the direction the fix must NOT change (a rank whose head event is still in flight still throttles the others). test_unified_mamba_pin_trace_581.py test_unified_radix_hicache_dispatch.py test_mamba_lock_ref_pairing_581.py 44 passed test_unified_radix_cache_unittest.py 851 failed, 29 passed, 656 skipped -- identical to the pre-change baseline for that suite (all pre-existing accelerator-required failures); the fix adds none. ruff, black and isort clean on both files. Live falsifier for the deploy: under the same agent load, ack_write must return to ~0 once the load stops, instead of freezing at 71/31/0.

…ot a live repair MY OWN CLAIM WAS WRONG AND sgl-project#609 WAS RIGHT. a9030fe states that HiMambaRadixCache "is the tree this rig actually runs". It is not. `registry.py:107` says the class has no construction site anywhere, and its module docstring says the default factory routes hierarchical + hybrid-SSM configs to UnifiedRadixCache. The rig DOES run mamba/GDN, which is what misled me, but that configuration lands in UnifiedRadixCache. WHY THIS IS WORTH A COMMIT RATHER THAN A NOTE SOMEWHERE. The module docstring already records three hardening commits that landed on this class BEFORE the reachability gap was found (sgl-project#581). sgl-project#609 exists precisely so a fourth does not arrive believing it is fixing production. a9030fe is that fourth commit, and leaving its claim unqualified in the tree would retire sgl-project#609's warning by contradicting it. WHAT DOES NOT CHANGE. The defect analysis, the fix and the 17 tests stand: the participation vote genuinely reached only two of the three trees, and this class genuinely lacked it. It is kept on the same terms as its three predecessors, under sgl-project#609's explicit decision to retain rather than delete. WHAT DOES CHANGE. There is no live instance and therefore NO GPU ARM: no boot can enter a class nothing constructs. The metal-proof request filed for a9030fe is withdrawn (/spinning/gpu-arb/requests/, same file, annotated rather than deleted so the reasoning survives). The claim in that request that this blocked the HiCache-file-backend mitigation rollback is withdrawn with it. THE LIVE PATHS WERE ALREADY COVERED BEFORE EITHER COMMIT: UnifiedRadixCache carries the vote from sgl-project#580 and HiRadixCache from sgl-project#610. A reader after a production behaviour fix wants unified_radix_cache.py. Tests unchanged and re-run: 17 green.

…e pipeline's dependency chain THE DEADLOCK. `_count_ready_acks` MIN-reduced a ready count across the group, and that reduction sat inside `check_hicache_events` -> `_get_new_batch_prefill_raw` -- the PER-MICROBATCH path of a pipeline, whose stages are at DIFFERENT offsets by construction. Measured 2026-08-17: PP0/PP1 inside the drain while PP2 was blocked in `_pp_recv_proxy_tensors`, waiting for data PP1 would only send after leaving the collective PP1 could not leave without PP2. A circular wait between adjacent stages -- the sgl-project#633 shape one level up, with a collective in place of a handler. The bug was PLACEMENT, not participation: every stage calls the function, just never on the same tick. WHAT THE REDUCTION PROVIDED, and where each part went: * OP-SEQUENCE UNIFORMITY -- concerned the TP all_reduce, which ran only on pp_rank 0. With no reduction there is no op left to keep in sequence. * THE EMPTY-QUEUE SENTINEL (sgl-project#581: one idle rank froze the drain on ALL ranks and `protected` ratcheted until the state pool died). Rank-local counting removes that failure mode BY CONSTRUCTION -- an idle rank cannot freeze a peer that no longer waits on it. * THE THROTTLE. Pacing, not correctness, and the one thing genuinely lost. CORRECTNESS IS OWNED BY THE MARKER, NOT BY ACK LOCKSTEP. The cross-rank hazard was publishing a shared content key incomplete on some stage. sgl-project#706's `PageCompleteness` bounds that per page: production is layer-sharded while storage is token-sharded, so slots arrive from several PP stages and `is_complete()` gates use. Ranks arbitrarily far apart therefore yield an INCOMPLETE page -- a MISS, never wrong bytes. The docstring now says this so nobody resurrects the collective for a service the marker owns. THE THROTTLE IS FILED, NOT GUESSED. No backpressure bound is shipped: one chosen without an operating point is another shipped-number-without-evidence (#505c). Instead a rate-limited drain-depth line (per-rank ready/pending at drain) makes the first real fast-rank host-pressure specimen attributable when it appears. TESTS test_rank_local_ack_drain_737.py, 8 passed, both arms: (i) deadlock -- ranks answer independently and may differ; an idle stage cannot freeze a busy one; and a CAN-FAIL that stubs torch.distributed to raise on ANY access, so a future edit that reintroduces a group op here fails loudly. Mutation-proven: adding one torch.distributed reference turns it red. (ii) divergence -- through the REAL `PageCompleteness`: worst-order arrival (PP2, PP1, then PP0) stays a miss until every slot is marked; order and tick are irrelevant, only completeness; a fast rank racing ahead cannot complete a page alone; double-writing a slot is refused. THREE PRE-EXISTING TESTS CORRECTED, both causes recorded rather than absorbed: * two sgl-project#734 fallouts I had missed -- the timeout stubs raised INSTANTLY, which the new dead-peer discriminator rightly reads as transport failure rather than expiry. A real timeout raises AT its deadline, so the stubs now sleep the bound. That keeps both the discriminator and the tests' original intent. * the sgl-project#581 throttle test asserted exactly the property this change withdraws. It is rewritten to the new contract -- a slow rank no longer holds the others, which is what sgl-project#581 actually needed -- with the withdrawal stated in the docstring, not silently deleted. Also: the sgl-project#580 comment at scheduler.py:6253 now states its ACTUAL guarantee -- uniform entry WITHIN the function once a rank calls it, which says nothing about all ranks calling it on the same tick. The ack collective leaned on it and deadlocked; anything added there needing GROUP agreement needs a pipeline-aligned point, not that one. REGRESSION mem_cache + managers: 19 failed / 3696 passed against a captured baseline of 19 -- ZERO new failures. Ruff: unified_radix_cache clean, scheduler.py 96 before and after (pre-existing).

…es, ledger sized, arm ticketed Three desk verdicts and the boot ticket. One of them retracts my own recommendation. (1) sgl-project#609 RECONCILIATION -- the UNREACHABLE marking on HiMambaRadixCache HOLDS, and is stronger than "the switches are off". A hybrid-SSM model under --enable-hierarchical-cache is routed elsewhere entirely: registry.py:107-112 returns _create_unified_radix_cache, with the comment "HiMambaRadixCache has no construction site anywhere". The only live construction is registry.py:133 MambaRadixCache(params). This RETRACTS my step-7 recommendation as written. I proposed booting --enable-hierarchical-cache "because it already tiers mamba state to host", citing hi_mamba_radix_cache.py:373/:553. Those lines exist but sit in a class that is never built -- precisely the sgl-project#581 aiming error the sgl-project#609 marking exists to prevent. Right by accident, wrong by citation. The mechanism survives in the class that IS reachable: the unified tree's mamba component treats a host-backed node as a valid match (mamba_component.py:71-74) and triggers load_back (:139-144), with _mamba_pool_host set when HiCache is enabled (:62). The arm is re-aimed, not abandoned. (2) DISK REACH -- mamba state is NOT host-RAM-terminal. PoolName.MAMBA is first-class across the storage layer (mooncake_store.py:730, storage_hf3fs.py:596) and the plain FILE backend carries it by key: hicache_storage.py:808-828 has _is_shared_mamba_key plus a canonical_mamba_blob mode, and _get_component_key/_sharded_path write it generically. sgl-project#555's contribution is already wired there as canonical_mamba_blob (full-width blob vs per-rank shard). No new disk path is needed; cross-stage sharing, if ever wanted, is that gate, not a writer. (3) HOST-LEDGER -- from the live checkpoint config: temporal 786432 + conv 30720 = 817152 elements per layer per slot, fp32 = 3.1172 MiB. Live topology is pp_size=3, tp_size=1 (PP, so whole layers at full width, no head sharding), 48 GDN layers split 17/16/15. Host post at hicache_ratio 2.0 (24 slots): 1272 / 1197 / 1122 MiB = 3591 MiB (3.51 GiB) total. Device pool at 12 slots is 1796 MiB. Step (5) sizing falls out of the same figure: 12 -> 6 slots returns 318/299/281 MiB per stage (~898 MiB), independently confirming the ~0.9-1.2 GB estimate. Framed as raising the binding stage's token ceiling, never as a KV share rising. WINDOW_TICKET_745 is the turnkey arm: add exactly three flags (--enable-hierarchical-cache, --hicache-storage-backend file, --hicache-storage-file-path), change nothing else, and accept on three log lines -- hit-rate lift against the 4-in-121 baseline, the HOST-LEDGER post against 3591 MiB, and at least one resume from a host-backed node (without which a lift could be device-side retention alone and proves nothing about tiering). The slot-floor change is deliberately a SECOND arm: it resizes the KV pool and would confound arm 1. Docs only. Nothing boot-verified.

…e lineage that actually runs The floor in mamba_pool_floor.py is only a guarantee if something bounds what cache retention may pin. That bound existed in exactly one place -- HiMambaRadixCache._mamba_pin_budget -- and registry.py:107 records that "HiMambaRadixCache has no construction site anywhere": every hybrid-SSM boot under --enable-hierarchical-cache is routed to UnifiedRadixCache. So the floor was charged at boot while the runtime ran unbounded. Two independent protections were unreachable on that lineage: 1. THE PIN BUDGET. UnifiedRadixCache.write_backup took inc_lock_ref(node) with no budget check. With write_through the threshold is 1, so every inserted checkpoint was pinned the moment it was created; acquire_component_lock takes a MAMBA ref on any node carrying a value, so each pin froze a state slot, and the eviction walk skips pinned nodes. The pinned set could therefore ratchet until it owned the pool and a REQUIRED allocation had nowhere to go. That is sgl-project#581 verbatim, and it explains why raising --max-mamba-cache-size only ever bought time: the ratchet scales with the pool. sgl-project#743 already COUNTED these pins for the trace line (_mamba_pins_in(ongoing_write_through)) without ever bounding them -- a counter with no actuator. The counter is reused rather than duplicated so the trace and the bound cannot disagree. 2. EVICT-BEFORE-FAILING. bind_tree_cache was called only from MambaRadixCache.__init__. Unbound, HybridReqToTokenPool.tree_cache stays None, so _alloc_mamba_slots_or_evict never evicts and retries -- the pool reported exhaustion while cached, EVICTABLE checkpoints sat in the tree. The #639b rank-parity tombstone branch hangs off the same handle and was equally unreachable. The budget is derived in mamba_pool_floor.mamba_retention_pin_budget so both lineages read one function; the previous split is what let a lineage ship without it. Refusing a backup costs at most a host-tier miss: the state stays cached on the device and stays evictable. Refusals are counted and rate-limited into the log, because a sustained count means the ack drain is not keeping up with insert pressure. Measured, not assumed, on boot_798_0822_0646 (Qwen3.8-27B-INT8, PP3): the floor for that config is 16, not 24 -- mamba_radix_cache_strategy is no_buffer so the ping-pong term is 0, and mamba_slot_reorder is on, so per_req = 1 active + 0 ping-pong + 1 donation/pin. The pool of 24 sits 8 slots above the floor, which is exactly the budget that was unenforced. TESTS: test/registered/unit/mem_cache/test_mamba_pin_budget_live_773.py, 11 tests, hermetic (CPU-only, no DMA controller). Red-first, and the can-fail proof for each gate was EXECUTED, not asserted: * removing bind_tree_cache -> 3 tests fail * disabling the guard in write_backup -> 1 test fails The second mutant initially SURVIVED, because the other tests drove the predicate directly rather than write_backup; TestWriteBackupActuallyConsultsTheBudget exists because of that survival and asserts on control flow. Both directions of the sgl-project#581 argument are covered: unbounded pins are shown to starve a required alloc on this lineage today, and a zeroed floor is shown to make that starvation reachable again through the guard -- so the safety is a property of the floor, not of the guard. REGRESSION: test/registered/unit/mem_cache/ = 1449 passed, 8 failed. All 8 proven pre-existing by an identical run on an untouched HEAD worktree (test_acceptance_emitters_758, test_localslot_family_756, test_mamba_anchor_seams_747), diffed name-for-name before claiming it.

…y where the lineage delivers it sgl-project#755 reduces the floor from 1+P+1+1 to 1+P+1: the donated slot BECOMES the next pin, so the two terms share one slot. The mechanism is real, gated per node, and tested -- but its config gate asks three questions about the CONFIG and none about the LINEAGE, and those select for opposite worlds. The gate requires enable_hierarchical_cache, because only a write-through host tier can promise the released anchor still exists. But registry.py routes a hybrid-SSM model WITH hierarchical cache to UnifiedRadixCache (:107-111), and MambaRadixCache -- the only class implementing the reorder -- is reachable only at :133, i.e. only when hierarchical cache is OFF: hierarchical=False -> reduction NOT taken (floor 24) | MambaRadixCache HAS impl hierarchical=True -> reduction TAKEN (floor 16) | UnifiedRadixCache NO impl Inverted in both rows. CacheInitParams.mamba_slot_reorder is filled from that same predicate on every boot (kv_cache_builder.py:237) and read only by mamba_radix_cache.py:525 -- always False where it is read, always ignored where it is True. The unified lineage's real demand was measured against the code, not assumed, and it is the FULL 1+P+1+1: * replacement slot allocated before the donation -- mamba_component.py:565 (_alloc_mamba_slot), called at :784/:757/:793 before donate_mamba_ping_pong_slot (memory_pool.py:1823), so a request transiently holds active + donated; * resume-anchor pin -- unified_radix_cache.py:1103, inc_lock_ref(new_last_node) right after insert at :1081, reaching MambaComponent.acquire_component_lock (mamba_component.py:491), which is what makes a STATE SLOT unevictable; * grep for mamba_slot_reorder across unified_radix_cache.py and unified_cache_components/ returns zero hits; MambaComponent's donate flow performs the un-reordered sequence regardless of the param. This was not cosmetic. schedule_policy.py:795-801 sizes the ADMISSION budget from mamba_slots_per_running_req for this same lineage (the branch keys on HybridReqToTokenPool, which is the token pool for BOTH trees; the "non-unified" in its comment refers to --enable-unified-memory, a different axis). So the standing boot charged 2 slots per request at both the floor and admission while the runtime held 3: a live one-slot-per-request under-floor, which is the sgl-project#581 direction -- the boot validates a pool the runtime over-draws and the shortfall surfaces late. The fix is a lineage predicate, and the capability is a module CONSTANT, not a config flag: it describes what the code can do, and an operator must never be able to assert it. Porting the reorder into the unified lineage flips UNIFIED_LINEAGE_IMPLEMENTS_SLOT_REORDER and the reduction returns with no other edit. Also adds the boot instrument that would have caught this: _validate_max_mamba_cache_size returns SILENTLY for any pool at or above the floor, so the floor only ever reached a log by REFUSING a boot. A pool sitting exactly on the floor looked identical to a comfortable one. The new MAMBA-FLOOR line states pool, floor and retention budget once at construction, and says plainly when the budget is 0. CONSEQUENCE, stated rather than left to be discovered: on the standing boot (--max-mamba-cache-size 24, --max-running-requests 8) the honest floor is 24, so the pool sits exactly ON it and the retention budget is 0 -- every mamba write-through backup is declined. That is the true posture, not a new restriction: the pool was always fully committed to the running set, the accounting just claimed otherwise. Buying cache retention means raising the pool, or implementing the reorder so the floor honestly falls to 16. The three sgl-project#755 tests that asserted the reduction from config alone now assert it under an explicit lineage patch. They test the MECHANISM's arithmetic, which is unchanged; what moved is where it is reachable. TESTS: test_mamba_reorder_lineage_773.py, 6 tests + 3 subtests. Can-fail EXECUTED: removing the lineage check fails 2. The original three gate conditions are each shown to still refuse on their own, so the gate was narrowed and not silently widened. Direction is asserted explicitly: the refused reduction moves the floor UP by exactly one slot per running request. REGRESSION: test/registered/unit/mem_cache/ = 1455 passed, 8 failed, the same 8 as an untouched HEAD worktree, diffed name-for-name.

…d-pinned number THE ACTUATOR. The earlier sgl-project#773 commits made the floor honest and made the write-through pin bound real, but they moved no bytes: the boot still pins --max-mamba-cache-size 24, so a corrected floor of 16 just sits underneath an unchanged pool. A counter with no actuator, which is exactly the shape this task exists to remove. Dropping the pin alone does not work, and that is the trap that kept it there. With the pin gone the sizing falls through to the --mamba-full-memory-ratio branch, which takes 0.9/1.9 of post-weights VRAM and produces a pool far LARGER than 24. The pin was compensating for a fall-through with nowhere good to land. The demand path already exists, is already floored at mamba_hard_floor (sgl-project#581) and already fitted to the budget (sgl-project#307). It was simply gated to uneven-DCP boots -- and the PP phase runs dcp_size 1, so the standing boot could never reach it. That single unreachable gate is why a VRAM number had to be chosen by hand on a rig whose planner is supposed to be the only VRAM authority. The widening adds a second route rather than replacing the first: size by demand when the operator STATED the concurrency. The predicate reuses the distinction _auto_mamba_target_concurrency already draws and already trusts -- a user-supplied --max-running-requests is a stated demand, an auto-defaulted one is not (the speculative hook resets an unset value to 48, and sizing to that over-provisions several GB and OOMs at pool init). Gate and target therefore cannot disagree about which numbers are real. A boot that never stated a concurrency keeps the fraction path, byte-identical. Derived result for the standing boot (mrr 8 stated, reorder on, ratio 2): floor 16, pool 20, retention budget 4. Cheaper than the hand-pinned 24 AND with real headroom above the floor for cache -- where pinning 16 by hand would have zeroed retention. 4 slots x 37.4 MiB = ~150 MiB on PP0, the binding rank, once the boot stops passing --max-mamba-cache-size. The number is now DERIVED. Retiring the pin is a boot-config change and is deliberately not made here. TESTS: test/registered/unit/model_executor/test_mamba_pool_from_floor_773.py, 9 tests + 4 subtests. Can-fail EXECUTED on both halves of the new gate: * removing the widening -> 1 fails (standing boot unreachable) * dropping the user_set check -> 2 fail (an auto-default would size) The second mutant first reported GREEN because ruff had reformatted the expression and the patch text silently did not match; the mutation step now asserts its target is present before believing a result. Recording that because a false green in a can-fail proof is worse than no proof. Each pre-existing refusal is shown to still refuse on its own (pinned size, explicit fraction, disabled radix), so the gate was narrowed-then-widened deliberately and not loosened by accident. REGRESSION: test/registered/unit/model_executor/ = 745 passed, 15 failed; the same 15 on a control worktree with my commits reverted, diffed name-for-name, and the +9 are these tests. mem_cache unchanged.

ROOT, corrected against the handover's framing. The handover read the store census as "15457 KV pages vs 11 mamba anchors = grid behaviour" and ordered the anchors published at every write-through. Measured on the same boot (boot_855_1028fence), that premise is a PAGE-vs-NODE denominator mix and the ordered fix is already in place: * `#969H BACKUP` = 33 lines = n=1..11 on EACH of PP0/PP1/PP2 at identical timestamps. The probe logs every call up to n<=40, so 11 lines means `write_backup` reached its component loop exactly 11 times per rank in the whole run -- and all 11 carried `mamba_value=has_value`. => mamba coverage OF THE HOST-BACKUP PATH is 11/11 = 100%, not 11/15457. * All 11 `.mamba` hashes in the store are also full-KV page hashes (intersection 11, mamba-only 0): one anchor per node at a real page boundary, shared key namespace. * The 15457 KV pages are the PAGES of those same 11 nodes: `write_backup_storage` writes `keys=node.hash_value` (every page of the node) while the mamba branch writes `keys=[node.hash_value[-1]]` (one trailing page). The real hole is one link earlier and is not mamba-specific: `_inc_hit_count` returns before any backup when `chunked=True` (upstream's "skip the hit count update for chunked requests"). Under chunked prefill NOTHING is published per chunk -- so the 11 backups are the 11 FINISHED requests, and a chunked prefill that never finishes publishes nothing at all. That is why the 13179-token prompt found its deepest anchor at 3072 and recomputed the remaining 10107 tokens: the anchors are as dense as finished requests, not as dense as chunks. FIX: allow a chunked-prefill node to reach the host tier, gated structurally on (storage tier present AND a MAMBA component present). The per-chunk node already carries a donated state on the device (`MambaComponent.prepare_for_caching_req`, is_finished=False branch); this early return was the only reason it never reached host or storage. UPSTREAM-MINIMAL: the chunked skip IS upstream (`hiradix_cache.py`), so this is a DEVIATION and carries its burden of proof. Named: (1) drain-and-flip -- a chunked prefill interrupted by a flip never reaches `cache_finished_req` (sgl-project#856 removed the carry, the flip DISCARDS it), so upstream's publish-at-finish never fires; (2) GDN hybrid -- a recurrent state is valid at exactly one token position, so the per-chunk anchor has no pure-attention analogue. Gate off = byte-identical to upstream (verified: branch truth table, 0 mismatches over all 8 chunked/write_back/force combinations). LAWS: `mamba-per-knoten-nicht-gitter` in its own words ("states per radix node/chunk like KV pages"), and it waives write volume explicitly. `kein-doppel-prefill` (sgl-project#939): loss bound becomes ONE chunk. sgl-project#968 PP0 DEBT -- WHERE THIS PATH STAYS RANK-LOCAL: the publish decision is taken at the scheduler's chunk boundary, which every rank runs for the same request at the same split, so it is unanimous BY CONSTRUCTION, not by agreement; this path holds no reduce (the `check_prefetch_progress` MIN is TP-scoped and the boot runs tp_size=1/pp_size=3, so it is structurally skipped). `raenge-nie-uneins` is met by construction and `#1028P CHUNK-PUBLISH` is how the claim gets CHECKED: identical n at identical timestamps across ranks, the evidence shape `#969H` gave for the 11. The one rank-local input reachable from here is the sgl-project#581/sgl-project#773 write-through pin budget, which fired ZERO times in that boot (trap-safe: bare 0, genuine 0) because 11 backups never approached it. Per-chunk publishing makes it reachable for the first time; `pin_skipped` rides on the same line so a nonzero count next to a divergent n is the divergence, named in advance. COST, QUANTIFIED NOT BUILT: one `.mamba` blob is 78446592 B = 74.8 MiB (measured, all 11 identical); a KV page is 32768 B. A published 4096-token chunk therefore adds 74.8 MiB of anchor on top of 128 MiB of KV (+58% L3). Host RAM is UNCHANGED -- the host mamba pool is pre-sized at boot from `hicache_ratio` (1.5) x device slots, so anchors roll through a fixed tier and land on disk. The int8-anchor idea (sgl-project#1013) would cut the 74.8 MiB and is deliberately NOT built here. ALSO: #1028B FETCH CAP instrument at the `min` in `hicache_storage.py`, the cap that decides how much of an existing KV prefix a prefetch may claim. It printed nothing: `final_pages`, `kv_pages`, `boundary=`, `hit_pages` each occur 0 times in the whole 5.87 MB log, so "anchors too sparse" and "KV prefix too short" produced the same number and were NOT separable from that boot. Now both terms print on one line. Desk checks (error-class matched): ruff F821/F811/F841/E9 clean on both changed files (new names + new attribute reads); branch truth table executed, gate-off equivalence exact. The gate is deliberately NOT memoised -- `enable_storage` is False at __init__ and only set in `init_hicache`, and a memoised early False would leave a wired-but-inert write path, the sgl-project#742/sgl-project#745 class this area has produced before. NOT YET BOOT-PROVEN. Acceptance is the next boot.

…lders; the cutover moved one WHAT WAS RED (parent a890b8a, test_1201_phase_stamped_handles.py): 13 failed, 3 passed. The 3 green are the default-path parity cases (an unowned cache still answers with its constructor pool; no counter to carry is not an error; a plain ReqToTokenPool without the hooks is not an error). After the cut: 16 passed; with the two sibling files (test_1040_req_pool_per_phase, test_cutover_participants_859) 82 passed. THE DEFECT. ReqToTokenPool is stamped onto four holders at CONSTRUCTION and rebind_req_pool_for_cutover moves exactly one, the scheduler's: * UnifiedRadixCache.req_to_token_pool (unified_radix_cache.py:429) * the pool's tree_cache back-reference (memory_pool.py:1871), whose only two bind_tree_cache callers are tree-cache constructors (unified_radix_cache.py:514-515, mamba_radix_cache.py:502-503), so the incoming pool keeps tree_cache=None for the whole phase and the evict-then-retry at memory_pool.py:2019-2024 never arms -- the sgl-project#581/sgl-project#773 regression that unified_radix_cache.py:507-513 was written to close * the pool's layer_transfer_counter (memory_pool.py:1876) * FutureMap.pool (overlap_utils.py:339), NOT moved by this cut Rank-uniform, so no ballot, digest or MIN can catch it: every rank is wrong the same way. Both phases' pools hold the same row count, so the divergence lands IN RANGE. Loud half: common.py:1849 -> free_slot's double-return refusal (memory_pool.py:492-497). Silent half, which runs FIRST: common.py:1836 reads the wrong pool's req_to_token row and hands those indices to token_to_kv_pool_allocator.free(). THE CUT. (a) UnifiedRadixCache.req_to_token_pool is a read-at-use property onto a registered owner, following kv_session_offload.py:2921-2933, whose docstring names this exact hazard. Unbound = byte-identical fallback to the constructor pool, so no non-flip boot changes. (b) _restamp_phase_handles re-stamps tree_cache, the pool's back-reference and the layer-transfer counter onto the incoming pool at the seam. (c) assert_req_pool_identity refuses at the end of the cutover (phase_flip_runtime, after the sgl-project#719 HiCache rebind, outside its try/except) when any holder still names a different pool. (d) cutover_participants: two new REGISTRY rows (request_pool_phase_ownership, req_pool_back_references) and req_to_token_pool added to MUTATED_STATE -- a HANDLE the cutover REPLACES, next to ten quantities it recomputes. future_map is on the registry as an explained gap, not moved. BRIEF PREMISE FALSIFIED -- (c) as briefed cannot be built. "Turn the two silent returns at memory_pool.py:1995-1998 / :3992-3994 into refusals" would refuse the DEFAULT path. layer_transfer_counter is None on every boot without a hierarchical cache: it is only ever set by register_layer_transfer_counter, whose callers are all HiCache/cache- controller paths, and swa_memory_pool.py:124-125 registers None on purpose; register_layer_transfer_counter's own docstring calls mamba_transfer_frame =None "the historic no-wait behaviour, unchanged". So None is a legitimate steady state, not a defect. What IS a defect is the handle failing to cross the seam, and that is decidable where both pools are in hand. The refusal therefore moved to the seam and the carry was built; the join sites are untouched. ANCHOR DRIFT (three, all reported rather than worked around): * "phase_req_pool_binding.py:180 is the only runtime rebind" -- :180 is the def line at the tree; the assignment is at :230, moved by CUT C (a890b8a). devindex still prints :180 because its store is pinned one commit back. * memory_pool.py:3992-3994 -- the guard is at :3993-3994; :3992 is a comment line. * memory_pool.py:496 -- the enclosing method free_slot is right, but :496 is a fragment of the double-return message; the line every finish hits is :506 (free -> free_slot), and :492-497 is the guard. All other anchors verified verbatim at the tree: unified_radix_cache.py:429 /507-513/514-515, mamba_radix_cache.py:502-503, memory_pool.py:1871/1876/ 1995-1998/2019-2024, mamba_component.py:480, common.py:1780/1836/1847/1849, kv_session_offload.py:2921-2924, cutover_participants.py:92/314-325. MUTANTS (scratch copies in /tmp/wt1201-orig, all restored): 1 DANGEROUS DIRECTION -- bind_req_pool_owner stores None, i.e. the pre-cut cached reference, which answers wrongly instead of refusing. Killed by 4: test_a_bound_cache_follows_the_owner, test_the_cutover_leaves_cache_and_scheduler_on_one_pool, test_the_kv_read_sees_the_rows_the_running_phase_wrote (the silent one), test_a_clean_cutover_passes. 2 DANGEROUS DIRECTION -- bind_tree_cache re-stamp removed, so the pool's evict-then-retry stays disarmed silently. Killed by 2: test_the_incoming_pool_is_bound_to_the_tree_cache, test_a_clean_cutover_passes. 3 assert_req_pool_identity returns early (#505a shape). Killed by 2: test_a_cache_left_on_the_outgoing_pool_is_refused, test_a_pool_left_bound_to_nothing_is_refused. 4 counter carry dropped. Killed by 1: test_the_layer_transfer_counter_is_carried_to_the_incoming_pool. VERIFIED: red-then-green tallies above; ruff parity on all four touched files (unified_radix_cache.py had 1 pre-existing F401 typing.Sequence before and after, zero new); import smoke of all four modules; no external writer to a tree cache's req_to_token_pool and no __dict__ access to it (grep, none); UnifiedRadixCache has no subclasses (devindex symbol_profile), so the property cannot be shadowed. NOT VERIFIED: no boot. Whether the layer-transfer counter carry ever fires on the standing flip boot is UNMEASURED -- it depends on whether the assembler registers a counter on both stacks' request pools, which only a boot decides. MambaRadixCache.req_to_token_pool (mamba_radix_cache.py:498) is the same cached-reference shape and is NOT converted here. ANSWER TO THE COMPLETENESS CRITIC, recorded as asked: this registry axis covers HANDLES THAT MOVED -- objects the cutover replaces and every holder that cached one. It does NOT cover WHAT WAS STILL OWED at the arm: launched-unreaped passes, outstanding receives, ring slots, lap counters. Nothing here detects a debt; MUTATED_STATE's read-window axis and the new handle axis are both about identity, not about obligations in flight. That second axis remains unbuilt and unclaimed.

* fix(detokenizer_manager.py): remove truncated decoding output chara…

f9ce154

…cter '�'

Ying1123 requested a review from hnyls2002 July 3, 2024 08:56

Ying1123 force-pushed the main branch from d530a1c to c7709d3 Compare July 3, 2024 21:59

hnyls2002 reviewed Jul 3, 2024

View reviewed changes

Titan-p closed this Jul 4, 2024

Navigation Menu

Uh oh!

* fix(detokenizer_manager.py): fix truncated decoded output - #581

* fix(detokenizer_manager.py): fix truncated decoded output#581
Titan-p wants to merge 1 commit into
sgl-project:mainfrom
Titan-p:main

Titan-p commented Jul 2, 2024 •

edited

Loading

Uh oh!

Titan-p commented Jul 2, 2024

Uh oh!

hnyls2002 Jul 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Titan-p commented Jul 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Titan-p commented Jul 2, 2024

Uh oh!

hnyls2002 Jul 3, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Titan-p commented Jul 2, 2024 •

edited

Loading