[Core] Complete snapshot recovery before serving

[matteso1]

Overview

Extract and complete the initialized-engine preparation/recovery sequence from vllm-project#51360. The snapshot CLI now validates recovered output before opening HTTP, using an internal session intended for reuse by ordinary serving (vllm-project#48193).

Draft for author review. Base 6e517b15c1, tested head 14903f61d1; the two commits separate mechanical extraction from behavior changes.

Claims

• Capture readiness follows completed rehearsal and communicator preparation.
• Communicator/memory recovery and a private output check finish before HTTP is bound. Failed or cancelled phases are terminal and retain a phase/error record.
• Parser/plugin validation happens before engine construction. Default ordinary-serving setup order is preserved.

Validation

Qwen/Qwen3-0.6B at c1899de289a04d12100db370d81485cdf75e47ca, FP16, context 512, memory target 0.50, RTX 4090 Laptop GPU, normal compilation and CUDA graphs.

Check	Result
MRV1 and MRV2: create, inspect, two restores across container restart	Pass on each runner; all 226 parameter digests unchanged, five one-token prompt outputs match each runner's ordinary reference exactly
Injected recovery failure after memory wake	Restore exits 1 before HTTP opens; phase/error retained and cleanup verified
Launcher test suite on Mac and Linux	110 passed, one existing AITER environment-contract failure reproduced on the unchanged image
Repository hooks	Pass, including mypy for Python 3.10

Unit command: .venv/bin/python -m pytest tests/entrypoints/launchers/test_launch_cli.py -q. Imported source paths/hashes were checked in each environment. Tests cover held preparation/recovery, canary mismatch, deadline/cancellation, partial-sleep failure, diagnostics and early parser validation.

GPU validation setup and CLI sequence

The changed Python files were overlaid on the official nightly image built from 593df82f, digest sha256:38450c4ab88bfeaf0097052f7996ad942c84287a95a2f6c2402fb9aabe007949 (Torch 2.15.0.dev20260928+cu130, CRIU 4.2.1, driver 595.71.05). Relevant call boundaries were compared and invoked file hashes verified. This is source-overlay qualification, not a full rebuilt wheel.

With the model cached and snapshot prerequisites configured, run the sequence separately with VLLM_USE_V2_MODEL_RUNNER=0 and =1:

vllm snapshot create Qwen/Qwen3-0.6B \
  --revision c1899de289a04d12100db370d81485cdf75e47ca \
  --tokenizer-revision c1899de289a04d12100db370d81485cdf75e47ca \
  --dtype float16 --max-model-len 512 --gpu-memory-utilization 0.50 \
  --snapshot-dir /e2e/artifact
vllm snapshot inspect /e2e/artifact
vllm snapshot restore /e2e/artifact --host 127.0.0.1 --port 8000

Stop/restart the same container, then restore again. The additional parameter checks and recovery fault were qualification-only instrumentation. Both comparison arms included the diagnostic-only runtime patch vllm-project#59661 at f8001354; it is excluded from this diff. The InfiniBand repair vllm-project#59699 was not applied.

Details

Previously the controller checked recovered output through HTTP after the listener was bound. SnapshotSession performs that check privately first and preserves the controller's public check. Memory wake can resume scheduling, so this helper requires exclusive startup ownership until recovery succeeds. It is not a live-server admission gate. Memory release/restore policy remains separate from quiescence and validation.

This is the existing CLI portion of the roadmap. vllm-project#54940 and its stack cover external-controller/device and transport plumbing; vllm-project#52132 captures the engine subtree while retaining its frontend. This draft shares the current CLI sequence without adding those control surfaces or an artifact manager. Start review in vllm/snapshot/engine.py, then follow its caller in vllm/snapshot/server.py.

Ordinary-serving integration, donor continuation, TP2 and relocation remain follow-ups. No latency/throughput claim: the runs were instrumented. KV-token capacity was ordinary/snapshot 54,048/54,032 for MRV2 (the existing CLI also had 54,032) and 51,680/51,472 for MRV1; the V1 difference remains unattributed.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment /ci run for upstream CI or /amd-ci run for AMD CI only whenever CI signals are needed.

Once the PR is approved or has the ready label, the PR author can also use the corresponding /ci run, /ci retry, and /ci cancel commands, or their /amd-ci variants. New commits do not start upstream CI automatically.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[matteso1]

Superseded by the upstream draft: vllm-project#59832