fix(checkpoint): deduplicate request-boundary payload pages

[yatesdr]

Summary

Fixes R28/R28.1 request-boundary LMCache write amplification without changing the boundary scheduler or model execution path.

• Replaces random generation-scoped payload keys with schema-v2 content identities.
• Reuses immutable attention pages across later prompt/response checkpoint generations.
• Keeps recurrent endpoint and target/draft auxiliary state keyed to the exact boundary.
• Makes exact manifest publication idempotent before SHM reservation or GPU copying.
• Omits GPU-to-SHM copies for content already readable in L1.
• Retains schema-v1 manifest/object restore compatibility in a separate object namespace.
• Preserves all-rank atomic publication and all-or-miss retrieval.

This is intentionally boundary-mode only. It does not select or depend on aligned checkpoint scheduling.

Stacked on #62 because the atomic recurrent checkpoint transport is not yet on dev.

Fixes local-inference-lab/vllm#716.

Root cause

The R28.1 connector assigned every boundary a random generation and included that generation in every target/recurrent/draft object key. A later turn therefore rewrote the complete historical attention state. INSERT OR REPLACE updated the manifest but did not make those generation-scoped payloads reusable.

Measured on the released R28.1 image before this fix:

• Exact 32K replay: 60 new payload files / 413.7 MB.
• Exact 197K replay: 224 new payload files / 1.544 GB.

Implementation

The new schema authenticates each page with the cache namespace, exact token-prefix chain, cache-group semantics, physical rank, and storage group.

• Full attention pages use their page-end token digest, so unchanged history has the same immutable key in every later generation.
• A partial final attention page uses the exact current endpoint and changes when its contents change.
• Recurrent groups use the complete boundary token digest because they contain endpoint state.
• Auxiliary state additionally separates checkpoint kind and draft-prefix length.
• The complete manifest still receives its own deterministic ID and is published only after every rank acknowledges its new or reused pages.
• An existing readable page is represented by a store-only no-copy marker. Retrieval rejects such markers and still requires every payload page.
• If content was evicted from L1, the producer may repopulate it, but the filesystem adapter's existing-key publication check prevents duplicate disk writes for the same content key.

The 200K content-key path reuses hash-chain endpoints. Its added median CPU cost is 0.220 ms; the pre-existing token-root construction is 12.986 ms.

Validation

Exact image base: voipmonitor/vllm:jovian-judgement-community-20260908-r28.1 (sha256:52ef7badcc33918f276d778d29bd972a798297584ba776476c7c09b7bdb50e5f). TP4/DCP4, MTP3, FP8 KV, full-and-piecewise graphs, scheduler budget 4096, fairness/interleaving enabled.

Correctness and durability

• 69/69 identity, index, real SHM/storage/RPC, and vLLM semantic-transfer tests pass.
• Deterministic 8K request: cold 248.3 MB; replay 0 files / 0 payload bytes on each of two repeats.
• After both services restart: identical output, 100% external-prefix hit, 0 new payload files / 0 payload bytes (4,120 bytes of SQLite metadata growth).
• 31.9K conversation plus six ~1K-token turns: cold 413.7 MB; later turns 303.3–441.3 MB each, independent of accumulated 32–38K history. The varying delta is endpoint state plus newly completed/partial attention pages, not historical attention KV.
• Missing or partial payloads retain the existing safe miss/recompute behavior.

Matched performance

RAM-only A/B isolates model/scheduler performance from NVMe activity. Same R28.1 base and launch configuration; only this patch differs.

Metric	Unpatched boundary	Patched boundary	Delta
32K prefill, warmed median	9,412 tok/s	9,410 tok/s	-0.02%
C1 MTP verifier	76.5 steps/s	76.8 steps/s	+0.4%
C8 MTP verifier	216.2 steps/s	215.4 steps/s	-0.37%

Speculative output-token rates are not used for the regression decision because MTP acceptance varied between runs. Verifier execution and prefill show no attributable regression.

Compatibility

Schema-v1 manifests remain readable with their generation-scoped v2 object keys. New schema-v2 manifests use content-addressed v3 object namespaces, preventing old/new payload mixing. The first store of an already-cached schema-v1 prefix may republish it once in the new format; subsequent generations reuse content keys.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)

• dev/*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 12a65d05-97ce-4945-9842-aed5dacb8528

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

The exact signed commit is now included in the #62 atomic-checkpoint transport branch, preserving Derek Yates’s authorship. GitHub marked this PR merged into that feature branch; this is not a merge into dev. #62 is the canonical review target for the combined transport and immutable-payload deduplication. The composed R29 runtime passes 138 checkpoint/storage/engine-driven CPU tests; GPU serving and restart qualification is still pending.