[Core] Add startup snapshot capture to ordinary serving

[matteso1]

Overview

Allow ordinary vllm serve to prepare an initialized engine for external capture, then activate the original process or a restored copy through the normal OpenAI-compatible server. This gives capture systems a path through the standard serving entrypoint, including its authentication and routes.

Depends on #59832 and builds on #51360. This draft targets upstream main, so the PR includes the two foundation commits from #59832. The ordinary-serving follow-up is commit 9027bd5190; its incremental diff changes eight files. Related design: #54940.

Claims

• Add opt-in startup capture to the ordinary Python launcher. The engine remains private and its public socket unbound until recovery and a private generation check complete; normal application construction follows.
• Reuse the shared snapshot session through a file adapter with explicit donor/copy activation, native-completion acknowledgement and a fresh challenge. Stale/invalid control and partial recovery failure terminate the startup instance with phase/error diagnostics.
• On the recorded single-GPU fixture, both model runners serve correctly from the continuing donor and two sequential restores after donor shutdown, including authenticated, concurrent and streaming requests.

Validation

Tested head: 9027bd5190. RTX 4090 Laptop 16 GiB, driver 595.71.05, CRIU 4.2.1, Qwen3-0.6B revision c1899de289a04d12100db370d81485cdf75e47ca, FP16, TP1, maximum context 512, memory utilization 0.50 and normal compilation/CUDA graphs. The runtime is a cached official nightly with recorded Python source overlays and loaded-source hashes, not a rebuilt current-main wheel.

Check	Result
Continuing donor, shutdown, two restores	Pass on both model runners
Model output	Five one-token prompts match ordinary reference text/tokens; log probabilities within 0.001
Normal serving	Missing/wrong API keys return 401; concurrent 16-token requests, a 16-token stream and eight 32-token requests at concurrency two complete
Integrity	All 226 named-parameter digests match across seven observations per runner; all 262 immutable image hashes remain unchanged
Recovery error after memory wake; stale release	Both runners exit without an observed public listener; phase/error records and owned-process cleanup verified
Regression suite	Mac/Linux each: 135 passed, 1 known test_snapshot_environment_contract failure caused by AITER_SITUV2_A4W4, previously reproduced on the unchanged runtime
Repository hooks	Pass, including Ruff and mypy

Restores retain their donor's KV capacity. V1 ordinary/sleep-only/donor counts are 51,680/51,472/51,664; V2 ordinary/donor counts are 54,048/54,032. Memory budgets reproduce those counts, but allocation variability remains unresolved. The short serving workload is a correctness smoke; clean activation timing and throughput neutrality are not established.

Commands and native qualification boundaries

Regression command inside the qualified Linux container, from /tmp:

/e2e/.venv/bin/python -m pytest /probe/source/tests/entrypoints/launchers/test_launch_cli.py -q

The external acceptance controller was exercised with:

/e2e/.venv/bin/python /probe/native_controller.py donor
/e2e/.venv/bin/python /probe/native_controller.py copy-1
/e2e/.venv/bin/python /probe/native_controller.py copy-2
/e2e/.venv/bin/python /probe/native_controller.py fault
/e2e/.venv/bin/python /probe/native_controller.py stale

The controller and raw native logs are retained outside this repository. This draft adds launcher/control regressions to the existing test file; it does not add native donor/copy CI coverage.

The fixture uses fresh PID/network/IPC namespaces with preserved loopback addresses and activation-specific control/log files. Old namespace descriptors remain pinned during qualification, retaining the donor's TCP state. Leave-running CRIU removes temporary shared-memory remaps before returning, so the external capturer collects them during post-dump and preserves hard-link identity. V1 used the collector before its final hard-link refinement; a real CUDA/semaphore witness and the final V2 model run verify that refinement. Full-model V1 coverage of the final collector remains open.

Runtime image: sha256:38450c4ab88bfeaf0097052f7996ad942c84287a95a2f6c2402fb9aabe007949. The diagnostic runtime fix from #59661 was applied consistently in the fixture and is excluded from this feature diff.

Details

The initial profile is one local CUDA rank, one Python frontend and a dense unquantized generation model using the existing compact release/reload policy. Unsupported launchers, topology and model/loading combinations fail before worker construction. The default serving path keeps its existing startup ordering.

The opt-in configuration is --snapshot-config '{"mode":"startup","control_dir":"/absolute/private/empty/dir","timeout_s":300}'. The new documentation describes the activation/challenge protocol and the capturer's filesystem, namespace and stdio obligations. Capture, storage, placement and native CUDA completion remain external. A validated control status means private recovery validation passed; ordinary HTTP probing establishes public readiness.

This is the ordinary-launcher follow-up to #59832. It adds startup capture before public traffic, while #54942/#54943/#54946 propose a broader externally coordinated lifecycle and HTTP control API; #52132 captures an engine subtree while retaining its frontend. The configuration schema here is proposed for review alongside #54940. There is no new live-server admission API or artifact manager. Public-provider integration, concurrent clones, TP2 and another-worker restoration remain separate qualification work.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[mergify]

Documentation preview: https://vllm--59851.org.readthedocs.build/en/59851/