[TianDi101]

Summary

Ports the pure-SSD / multi-drive performance work from feat/umbp-pure-ssd-multidrive onto the backend-agnostic SSD path, then fixes the benchmark that measures it and flips direct I/O on by default.

The backend-agnostic refactor unwired SSD (Phase 0) and re-landed it as SsdBackend : MediumBackend (#542, #543). That re-landing was a clean-room implementation and did not carry the performance work forward, leaving a structurally correct but performance-naive tier: single drive, buffered I/O, single-threaded CRC, one device read per key, no read coalescing, no instrumentation.

Commits

bae92878 — perf(umbp): port the pure-SSD/multi-drive work onto the backend-agnostic SSD path

Below the backend boundary (ports near-verbatim):

• Record format v3 — records and values padded to 4096B, the precondition for O_DIRECT. Capacity is now charged in padded on-disk bytes, so a store of small values can no longer overrun capacity_bytes. kFlagNoCrc + KeyMeta::crc_valid keep a store written with checksums off readable by a process with them on.
• Direct I/O (UMBP_SSD_DIRECT_IO) with a startup probe that falls back to buffered on tmpfs/overlayfs rather than failing to come up.
• DropUnparsedTailsLocked — startup repair for stale-version and torn-tail records, which would otherwise make every later append unreadable.
• ShardedSsdTier + MakeFileSsdBackend — comma-separated UMBP_SSD_DIR fans one logical tier across N drives with capacity-aware placement and parallel per-shard IO.
• ParallelFor over UMBP_SSD_TIER_IO_THREADS for the CRC/assembly phases, matching the DRAM tier's 4-thread default.
• ssd_perf.h — the [SsdPerf/tier|peer] breakdown behind UMBP_SSD_TIMING.

At the backend boundary (adapted, not copied):

• Single-flight read coalescing in PeerSsdManager (memcpy fan-out only; the RDMA-WRITE push path depends on reverse-registration and SSD-specific batch RPCs that do not exist in the agnostic proto).
• PeerSsdManager::WriteBatch / PrepareReadBatch, with SsdBackend::BatchCommit / BatchResolve restructured from per-key loops into one-lock/one-batch/one-lock passes. Without this a multi-drive tier can never be saturated — the per-key path costs one device read per key and idles every drive but one.
• MediumBackend::Counters() — a defaulted-empty virtual, so SSD's metrics reach Prometheus without a dynamic_cast<SsdBackend*> in the metrics path.
• Hugepage-backed staging arena and UMBP_PEER_{MIN,MAX}_POLLERS.

Not ported, and why: UMBP_ROUTE_PUT_SSD_MODE / IsPureSsdMode() have no analogue — medium = SSD already makes a node an SSD put target, and the routing plane no longer ranks tiers. Round-robin put selection, the RDMA read fan-out push, and the copy-pipeline metrics remain open.

efd48551 — fix(umbp/bench): make umbp_bench actually configure the SSD tier it measures

Two bugs that silently invalidated any SSD number the bench produced:

1. os.makedirs(cfg.ssd.storage_dir) with a comma-separated multi-drive path created one directory literally named "a,b" instead of one per drive.
2. The bench builds UMBPConfig directly and never calls FromEnvironment(), so no UMBP_SSD_* knob reached it. direct_io in particular stayed off, and buffered reads can be served entirely from page cache.

Without the fix the "SSD" tier reported 95 GB/s, within 1% of DRAM. With it, 10.2 GiB/s end-to-end and 767 GiB of verified device reads on the host counters.

7591f8a7 — feat(umbp): make SSD direct I/O the default

The SSD tier is itself a cache, so buffered I/O gives it a second, unmanaged DRAM cache underneath — it evicts host memory the DRAM tier and the engine are competing for, it makes the node report DRAM it is consuming as free, and it makes any measurement of drive behaviour meaningless.

Safe as a default because the failure mode is already handled: SSDTier probes O_DIRECT at startup and falls back to buffered with a warning on a filesystem that cannot do it. Set UMBP_SSD_DIRECT_IO=0 (or ssd.direct_io=False) to deliberately exploit the page cache.

Test plan

• Build + 44/44 ctest green. The SSD tests need the ionic/libibverbs bind-mounts; without them 4 integration tests fail "no active RDMA device", which is environmental.
• New coverage: test_sharded_ssd_tier.cpp, test_ssd_direct_io.cpp, test_peer_ssd_single_flight.cpp.
• Measured on n06-21 (2x Micron 7450, 32 GiB dataset, 2 MiB values), no env var set: [SSDTier] reports direct_io=true, and host /sys/block counters show 96.0 GiB of device reads against 96.0 GiB of bench reads — every read hit the drive. 1 drive 5465 MiB/s, 2 drives 10233 MiB/s (1.87x end-to-end; 2.07x at the tier).

🤖 Generated with Claude Code