[luckyq]

Series map (Phase 1 harness, one PR per step; each is based on the previous one)

PR	Step	Test cases
#137	Harness core: stack, client, telemetry, reference engine, CI unit tests; phase1 job in vllm-e2e.yml on the two-GPU runner (TC-01; #138/#139 extend --cases)	TC-01 integration generation correctness
#138	Local DRAM flow and hinted peer transfer, hybrid geometry flags	TC-02, TC-03
#139	Guard protection, takeover, replacement	TC-04
#140	Streaming client, performance arms, Guard recovery timing	TC-05, TC-06
#141	Sidecar kvcr_service, 2P+1D, two nodes, k8s / Slurm / Docker runners; phase1-2p1d CI job (2P+1D on the two-GPU runner, prefill workers share GPU 0)	all cases on those topologies
#142	docs/ci.md: what the two CI lanes verify, how to run them elsewhere, how to read evidence	—
#143	Test plan, run history, bare-vLLM probes	—
#180	--engine sglang: dynamo.sglang + HiCache KVCR storage backend (quick-start config), SGLang telemetry mapping, deterministic mode; no Guard/disagg	TC-01..03, TC-05 on SGLang

Replaces #103. All seven are drafts until the per-PR GPU runs on the current main image are done.

Problem

• Part 5 of the stacked Phase 1 series (on top of TC-05/06). So far every case runs the aggregated two-worker layout in one container; the deployments that matter for KVCR (sidecar kvcr_service containers, 2P+1D disaggregation, two nodes) have no runner.

What this adds

• --external-services DIR: sidecar topology, kvcr_service runs in its own containers and shares sockets/logs through DIR.

• --topology disagg: 2P+1D on three GPUs; workers 0/1 are prefill workers with a MultiConnector (KVCR OffloadingConnector + NixlConnector producer), worker 2 the decode consumer; requests pin the prefill worker. The KVCR tier list, state-agent settings and self_describing_kv_events are mirrored at the MultiConnector top level because Dynamo read them only there (fixed upstream by dynamo#16778).

• --remote-worker INDEX=URL + --etcd: two-node runs through tests/e2e/phase1/node_agent.py (spawns the remote worker and its sidecar, relays logs); etcd discovery because Dynamo's file discovery does not cross nodes.

• Runners: scripts/docker/ (build the quick-start image from committed source with recorded Dynamo / vLLM-adapter revisions; run the harness in that image on two selected GPUs), scripts/k8s/phase1-pod.py (pod manifests: single or two nodes, overlays, extra env), scripts/dlcluster/ (Slurm sbatch + image build) with the hybrid matrix pinned to the measured Qwen3.5-4B geometry.

• Dockerfile.quick-start-vllm (rebased onto v0.2.0 pins): records the Dynamo / vLLM-adapter / KVCR revisions into /opt/kvcr-build/source-revisions.json so result.json carries image provenance; configurable runtime user for sidecar images.

• phase1-2p1d job in .github/workflows/vllm-e2e.yml: the 2P+1D layout on the two-GPU runner (--gpus 0,0,1 --gpu-memory-utilization 0.35, kvcr-service sidecars as background processes of the step), TC-01..04 + TC-06, functional verdicts only.

Validation

• Shared-GPU 2P+1D on dev-02 (v020c-2p1d-shared, 2×H100): TC-01..04 + TC-06 PASS, Guard promoted 1.0 s, first correct response 1.4 s, replacement serving 82 s.
• ruff, pytest tests/ci: see CI.
• GPU: 2P+1D single node and two nodes 6/6 with KVCR v0.1.1 + Retry a refused start_write once with metadata after promotion #107 (two-node TC-06 5/5 Guard hits, first correct response 1.5 s); findings and workarounds (replacement prefill needs --gpu-memory-utilization 0.35 because the decode worker keeps the dead prefill's KV mapped over NIXL) are in the test plan part of this series.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.