[TianDi101]

Two fixes found by the DRAM 300 GB → SSD 1.5 TB end-to-end run (InferenceY agentic, DeepSeek-V4-Pro, n06-21), where SSD-backed loads under the sglang UMBP linker crashed the scheduler.

1. A load batch larger than the SSD staging arena failed whole

Symptom: UMBP get failed … success=78/2664, with 2,124 staging arena busy; rolled back N reservations warnings, then sglang's scheduler died (the linker raises on any failed get after a hit).

Cause: the linker asks for ~2,000 keys per call. A resolve either fit the whole batch in staging or failed all of it. Local readers also held staging for the full 3 s read lease after their copy had finished, so the arena had to cover read rate × 3 s.

Fix:

• A local SsdBackend::BatchResolve stages up to half the arena per call (the first key always fits). The rest comes back kBusy.
• PoolClient serves local gets and ranged gets in parts (resolve → copy → drop the part's holds) until every key is served.
• Staging returns as soon as its last hold drops. It is reclaimed by the reaper, and on demand when an allocation finds the arena full.
• Existence probes use BatchContains, so they stage nothing.

2. A local read's hold could expire mid-copy

In-process readers were protected only by time: the SSD staging lease (3 s) and the DRAM/HBM eviction fence (500 ms). A copy that outlived its lease could have its staging slot reused, or its DRAM page evicted, while still reading it.

Fix: a local read holds its bytes by refcount. ResolvedEntry::hold is a shared_ptr that the reader keeps for as long as it reads. PageBackend slots and SSD staging spans each own one, and neither evicts, frees nor reuses what is still held. There is nothing to release and nothing that expires, so no token, generation or overdue state is needed. Remote readers keep the timed lease (#749 is the remote half and is not fixed here).

API change to review

MediumBackend::BatchResolve gains a 4th parameter ResolveReader reader = kRemote, and ResolvedEntry gains hold (kLocal only). The reader stays a parameter because three behaviours depend on it: no timed lease, partial staging (a remote reader retries the whole batch, so partial staging would livelock it), and #781's extents.

Metrics

• SSD medium event staging_expired_reclaim: staging spans freed after their lease and holds were gone.
• mori_umbp_backend_medium_state{tier="SSD",state="staging_pages_peak"}: the arena's highest occupancy since start. staging_pages_in_use is sampled per scrape and misses bursts.
• kBusy keys are not counted as resolve outcomes. A held-but-unreadable key is status="failed" instead of a miss.

Also

test_peer_pool's FailingEngine now overrides CreatePlanBuilder (#774's interface). Before this, test_peer_pool did not build on main.

Testing

• New tests: a local batch over half the arena is staged in parts; local staging lives as long as its holds; a remote lease outlives a local hold; a local hold fences PageBackend eviction until dropped; a ranged get of 20 objects through an 8-slot arena; resolve metrics count only final answers.
• test_page_backend 40, test_ssd_backend 35, test_peer_pool 41, test_component_metrics 22, test_umbp_ssd_ranged_io 7, test_umbp_local_first 4, test_umbp_no_master 9: all pass.
• End-to-end on the merged chain (main + fix(umbp): a local eviction round no longer rescans its queued demotions per batch #783 + fix(umbp): SSD tier gives disk back a segment at a time #775 + fix(umbp): an in-place SSD read counts as an access #784 + this + feat(umbp): SSD demote/promote skip staging; promoting reads land in DRAM; 4 GiB staging preset #776 + feat(umbp): a local ranged GET off SSD reads only its extent #781): results to follow in a comment.

🤖 Generated with Claude Code

[TianDi101]

End-to-end validation of the eviction PR chain: passed.

Branch test/umbp-evict-e2e-v2 @ 2e1663d91 = main (097bb470f) + merges of #741, #762, #763 and #683, nothing else. (The #683 merge drops PageBackend's local-evict counters, which #741 removed, and adds #763's K dropped summary line.) Workload: InferenceY single-node agentic (DeepSeek-V4-Pro fp4, TP8 + DP attention, MTP, conc 256, CI run 29816's config) on n06-21, 3600 s profiling. UMBP is a standalone server with no master (so default local eviction), with a 300 GB DRAM tier demoting to a 1.5 TB SSD tier on 3 drives.

	earlier e2e on #753	this chain (exact PR heads)
Input throughput	281,015 tok/s	283,993 tok/s
Prefix hit (HBM + UMBP)	92.03%	91.99%
Failed loads / SSD read or write errors / sglang crashes	0 / 0 / 0	0 / 0 / 0
DRAM→SSD demotions / copies dropped	1.48M / not measured	1.47M / 0
Demotion failures retried until they landed (#763)	not measured	5,048 (SSD staging full), all landed
SSD keys deferred to a later round / staging released early (#762)	2.93M / 60.3M	2.83M / 60.3M
Local eviction rounds / stalls (#741)	19 / 0	19 / 0

The log signatures of the 7 bugs this e2e originally found all stay at 0: whole-batch staging failures, UMBP get failed, overdue local holds, SSD write failures, PrepareReadBatch read failed, dropped demotions, and any [error] line. An earlier pass on the pre-trim heads gave the same result (277,392 tok/s, 0 failures).

Notes:

• The harness sets UMBP_STANDALONE_STARTUP_TIMEOUT_MS=1800000. Since fix(umbp): keep the standalone server serving when its GPU-registered DRAM pool is migrated #756 the server reports ready while its 300 GB pool is still being GPU-registered, and under concurrent model loading sglang's default 30 s connect timed out. That is not part of this chain.
• One rerun was lost to an sglang GPU memory fault in one DP rank (its own kernel, no UMBP data being loaded at the time); the rerun passed.

🤖 Generated with Claude Code