[TianDi101]

Turns the primitive benchmark into a CI gate, and removes umbp_bench.py.

umbp_prim_ci.py brings up standalone-process mode — a umbp_standalone_server, and for the second arm a umbp_master in front of it — runs umbp_prim_bench.py over 1/2/4/8 concurrent client processes, and turns the rows into a verdict. It runs in the existing umbp-unit-test job.

What is gated

Correctness, on every pass. All five primitives must report no failed operations and no read-back mismatches at every rank count. Verification now compares every key over its whole object, through the op's own call — so get_ranges is checked with its real offset and size tables, which is the only way a wrong range offset is catchable. The payload is a keyed pseudo-random stream; a repeated tag verifies clean against a shifted read.

batch_exists throughput must still grow with rank count. A control plane that serialises reads looks healthy at one rank and collapses at eight — no single-concurrency latency number can see that, and comparing absolute numbers across runs on a shared machine is noise. A ratio measured inside one run is not.

Why the floor is 2.0, and why the measurement is a separate pass

Measured through the correctness pass's geometry the exists timed window is under a millisecond, and three identical runs of one unmodified build scattered 1.59x / 1.88x / 3.49x, not even monotonic in rank count. A floor of 2.0 would have failed two of the three.

Run as its own pass (exists alone, 8192 keys/rank, 4 KiB objects, 100 passes) the same build gives 3.50x–5.39x over four runs, monotonic, one-rank baseline stable to 3%. So the floor sits ~43% under the worst healthy run seen and 2.7x over the regression it is aimed at. If this number ever reads noisy, raise --scaling-passes — a longer window — not the floor.

Verified against both arms of the 09876ba7 control-plane regression on one node, same geometry, same session:

arm	correctness	exists 1→8	verdict
a6970dc1	0 bad / 20 rows	4.30x	PASS
09876ba7	0 bad / 20 rows	0.65x	FAIL

0.65x independently reproduces the 0.75x recorded for that regression on a different node and geometry. Both arms pass correctness — it is a slowdown, not a functional failure, and the gate catches it on the scaling axis alone.

exists is the gated primitive because it is the only one of the five that moves no bytes, so its scaling is a property of the control plane rather than of how much memory bandwidth the runner had left. The other four are measured and printed but not gated: both a good and a regressed build have been observed to anti-scale on the writes, so a floor there would be either vacuous or permanently red.

Absent-key sweep

The probe is swept over the fraction of keys that were never written (0 / 0.5 / 1) — a resident key is resolved locally, an absent one is not. Only the 0 point is gated; the others have no measured baseline yet and report as info. Correctness is checked on those passes, so a probe answering True for an absent key fails the build — for a prefix-cache probe that is the more dangerous direction.

Note for reviewers: on the nomaster arm absent keys are measurably faster (1.08M vs 0.95M keys/s), because with no master a local miss is conclusive. The "absent keys cost a master round trip" argument applies to the master arm only.

Pool sizing is charged in pages

The DRAM medium is paged and hands out whole pages, so a 64 KiB object in a 2 MiB page occupies 2 MiB. Sizing by keys * object_bytes ran the pool 32x short; because the just-seeded keys are the least recently used, that surfaced as seeded keys never became visible rather than as anything about space — and it cost one false A/B result before it was found. pool_bytes() now charges ceil(object / page) pages per key and pins the page size on both arms.

umbp_bench.py removed

Its concurrent_perf subcommand is superseded by the rank sweep; the compound flags the rest depended on were deleted earlier. The same five APIs are exercised, but four measurements built on them do not come across:

gone	primitive bench instead
correctness: every key vs its full payload	now equivalent — this PR closes that gap
exists_perf: sweeps batch [1, 32, 128, n]	one --batch per invocation
batch_perf: --dest-layout shuffled defeats SGE coalescing	contiguous slots only
ranged_perf: ranged vs whole-object as a ratio per fetched fraction	times ranged I/O with no whole-object arm

Also gone: the DSv4-Pro pool presets, the --role writer/reader split, and the Mooncake comparison path the mooncake-vs-umbp-bench skill drove (that skill is now marked broken). The last committed version stays recoverable at git show 09876ba7:tests/python/umbp/umbp_bench.py.

Media selection — wired, but only dram works

--media dram|hbm|ssd|dram:70,ssd:30 is present and unit-tested, and generates a backend-policy file for mixtures (HBM is policy-only: Validate() demands hbm.capacity_bytes > 0 and no env var sets it). Only --media dram, the CI default, is verified end to end. ssd fails every put with [SsdBackend] Resolve: staging arena busy; rolled back 0 reservations — rolled back zero, with 256 free pages against a 16-key batch, so it is not arena sizing; AcquireStagingSpanLocked returns empty for page_count == 0, which points inside the backend. Mixtures load the policy but the server then reports medium=[DRAM ...] with no SSD backend. Both are recorded in the module header. Happy to split this out of the PR if you would rather it landed only once it works.

Tests

test_umbp_prim_gate.py replays both arms of the regression through the gate, so the floor cannot be retuned past what it exists to catch, and pins the page-charged pool arithmetic. test_umbp_prim_bench.py covers the key spaces, payload and absent-key mixing. Both are pure arithmetic — no mori, no server, no GPU — so they run in the ordinary pytest step. 44 new tests.

Cost

The job's timeout-minutes goes 30 → 60 and the run asks the server for a ~35 GiB pool at CI defaults. Worth a look if the runner is tighter than I assume.

🤖 Generated with Claude Code