[tjtanaa]

Overview

Enable AMD Qwen3.8-Flash-Next hybrid-cache restoration with UMBPStoreConnector, building on the embedded UMBP runtime in vllm-project#58568. This supports embedded DRAM offload without adding a new model implementation.

Claims

• Preserve per-group cache namespaces and per-layer transfer sizes for attention KV, compressed QSA keys, GDN state and PLE convolution state.
• Restore committed recurrent-state boundaries through asynchronous hybrid loading; reject incomplete restores and report completed backend failures at request level. In-flight native timeouts remain fatal.
• Enable QSA full-block restores with UMBP while rejecting unsupported connectors and partial-hash restores.
• Validate actual decode-graph replay after external restoration, including fresh store/restore after connector reset.

Validation

Local validation on MI355X, TP4 BF16, Triton MoE and FULL_DECODE_ONLY, using vLLM f05f7a22 and repaired MoRI cd9772b5. Prefill runs eagerly.

Check	Result
Native MoRI pointer suite	7 passed; old binary reproduced 2 reset failures
Four-GPU transfer suite	4 passed
Eager/lazy offload reset cycles	Both passed; each restored 7,840 tokens through 56 loads, zero transfer failures
Native/eager-offload/lazy-offload graph tests	All passed; 256 successful restored-decode replay calls across four ranks
Adapted single-needle retrieval, 8K–128K	Native 50/50, UMBP 50/50; identical paired answer text/token IDs
NIAH restore integrity	13,640 completed loads, zero failures; each needle inside restored coverage

NIAH uses five lengths, five needle depths and two cases per cell, with a pinned Paul Graham corpus and adapted RULER generator. These are focused retrieval results, not broad model-accuracy or performance claims. Graph-restored requests exercised batch size 1.

Earlier runtime validation at e4314a19: 239 CPU tests passed with one GPU-only skip, plus 16 focused GPU tests. These suites are historical coverage, not claimed as a fresh full-suite run at the final HEAD. Final changed-file hooks passed.

Details

The base runtime can disagree on grouped cache keys and transfer sizes, or restore recurrent state after input preparation. The change reuses existing cache specifications and committed boundary hand-offs, updates block mappings, and forces asynchronous loads for hybrid caches.

The validated request flow is:

1. Prefill and publish the committed cache/state prefix to embedded DRAM.
2. Clear GPU prefix-cache residency while retaining external entries.
3. Verify complete group/rank coverage and restore before preparing model inputs.
4. Continue decoding and replay the captured decode graph.

Full-block QSA boundaries allow the non-prefix-cacheable raw-key scratch ring to rebuild. Timed-out native loads retain destination ownership and fail the engine; this is not cancellable/recoverable I/O. Permanent native stalls still require process/container supervision.

Requires the separate MoRI no-master clear-acknowledgment fix: otherwise reset succeeds but later stores remain blocked by CLEAR_PENDING. Build instructions and the portable patch are tracked in docs/features/umbp_embedded_dram.md and tools/umbp/mori-no-master-reset.patch. Selected hybrid work is adapted from vllm-project#59275 commit 2d2615af.

Limits: text-only embedded DRAM, TP4 and full-block restores. Production QSA ordering is unchanged and repetitive-prompt numerical variability remains. TP8+AITER fails weight preparation even without UMBP. Prefill graphs, broader concurrent correctness, speculative decoding, multimodal and distributed-runtime configurations are not qualified. No speedup claim.

AI assistance was used for implementation, validation and this description. Human review remains required.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)