Repository navigation
Bench/umbp staging autosize - #564
Merged
TianDi101 merged 3 commits intoAug 14, 2026
Merged
Conversation
A remote SSD read stages the value into a registered host arena of N slots. A key that finds no free slot does not retry and does not block -- it comes back as a plain cache MISS (slot_full_rejects_; UMBP_SSD_GET_MAX_ATTEMPTS is described in doc/pure-ssd-mode.md but is not implemented), so the fixed 512-slot default silently capped every sweep wider than 512 keys: a 2048-key batch returned exactly 512 hits and 1536 misses per pass while still printing a healthy MiB/s, computed only over the keys that got a slot. The drives were never the bottleneck, so multi-drive scaling measured flat. Size slots from the largest batch in the sweep instead of a constant, and derive arena bytes from the slot count so slot size always covers value_size. The env vars remain as escape hatches but no longer fail silently: warn when slots are under the largest batch, when a slot cannot hold one value, and when the arena is at least as large as the dataset -- the inverse trap, where repeated passes are served from staging DRAM and never touch a drive. Measured on n06-21, 2048 x 2 MiB, 1 drive: 75% misses before, 0% after. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first pass sized the arena from the largest batch alone. That is only half of it: a staged page is freed when the reader's release lands and otherwise only when the read lease expires, so steady-state occupancy is roughly batch_width + (completed reads/sec x lease). Measured on n06-21 at 2304 slots: 1 drive, batch 128: 3042 req/s -> 1521 + 128 = 1649 slots -> 0% misses 2 drives, batch 128: 4699 req/s -> 2350 + 128 = 2478 slots -> 23% misses 1 drive, batch 2048: 2939 req/s -> 1470 + 2048 = 3518 slots -> 44% misses The throughput term is what makes this bite as drives are added: faster media completes more reads per second and therefore needs a BIGGER arena, the opposite of the intuition that one slow drive is the tight case. So a sweep that looks clean at one drive starts silently missing at two, and the drive scaling it was measuring is exactly what breaks it. Budget two batches of burst plus a fixed margin, and default the lease down to 50ms for SSD runs -- it only has to outlive one sub-millisecond RDMA round trip, so the 500ms default pinned thousands of pages purely waiting to expire. With both, a 512-wide sweep needs 2048 slots instead of an unbounded number. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ghput The previous sizing assumed a staged page is held for about the read lease, so batch + reads/s x lease bounded occupancy. Measurement says otherwise: on 4x KIOXIA CD8 at ~15k reads/s a 512-wide batch needs 4096 slots where that model predicts ~1400, implying ~200ms of hold against a 50ms lease. The RDMA round trip and the asynchronous release dominate, not the lease. So the constant is now calibrated at the fastest media available rather than derived (4 drives, batch 512, 32 GiB dataset): 2048 slots -> 20% misses, 13.4 GB/s reported (fiction) 4096 slots -> 0% misses, 30.0 GB/s 8192 slots -> 0% misses, 29.0 GB/s 8192 buying nothing over 4096 both confirms 4096 suffices and rules out cache inflation -- were staging serving the reads, doubling it would have raised the number rather than leaving it flat. With this, the homogeneous 4-drive curve is clean at every point (misses=0, within 0.3% of iostat): 10.3 / 18.5 / 24.9 / 30.3 GB/s for 1/2/3/4 drives. Note for deployments: a node serving ~30 GB/s needs ~8 GiB of pinned staging, where doc/pure-ssd-mode.md's validated example uses 512 slots (1 GiB). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TianDi101
added a commit
that referenced
this pull request
Sep 3, 2026
- add arena auto sizing for ssd backend benchmark
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Technical Details
Test Plan
Test Result
Submission Checklist