[luckyq]

Series map (Phase 1 harness, one PR per step; each is based on the previous one)

PR	Step	Test cases
#137	Harness core: stack, client, telemetry, reference engine, CI unit tests; phase1 job in vllm-e2e.yml on the two-GPU runner (TC-01; #138/#139 extend --cases)	TC-01 integration generation correctness
#138	Local DRAM flow and hinted peer transfer, hybrid geometry flags	TC-02, TC-03
#139	Guard protection, takeover, replacement	TC-04
#140	Streaming client, performance arms, Guard recovery timing	TC-05, TC-06
#141	Sidecar kvcr_service, 2P+1D, two nodes, k8s / Slurm / Docker runners; phase1-2p1d CI job (2P+1D on the two-GPU runner, prefill workers share GPU 0)	all cases on those topologies
#142	docs/ci.md: what the two CI lanes verify, how to run them elsewhere, how to read evidence	—
#143	Test plan, run history, bare-vLLM probes	—
#180	--engine sglang: dynamo.sglang + HiCache KVCR storage backend (quick-start config), SGLang telemetry mapping, deterministic mode; no Guard/disagg	TC-01..03, TC-05 on SGLang

Replaces #103. All seven are drafts until the per-PR GPU runs on the current main image are done.

Problem

• KVCR has no end-to-end regression harness: nothing in CI exercises the Guard, the test plan's TC-01..06, or a KVCR-disabled reference engine.
• This is part 1 of a stacked series (PR per test-plan step): core harness + TC-01. Follow-ups: TC-02/03, TC-04, TC-05/06, disaggregated and two-node topologies, CI job, docs.

What this adds

• tests/e2e/run_phase1.py + tests/e2e/phase1/: brings up a Dynamo frontend (KV router), two vLLM workers with KVCR tiers and kvcr_service Guard sidecars on two GPUs; runs a KVCR-disabled reference engine first (tokens must reproduce, worker log must be KVCR-free); every request's tokens are compared with the reference; KVCR telemetry is parsed from the vLLM metrics log; evidence is result.json + JUnit + per-request files. The service launcher picks kvcr.service.server (KVCR 0.2.0) or kvcr.kvcr_service (earlier releases) by what the installed package provides.
• phase1 job appended to .github/workflows/vllm-e2e.yml: after peer-transfer, same ECR image and prod-kvcr-tester-amd-gpu-2-v1 runner, run_phase1.py --mode guard --cases TC-01, evidence uploaded as an artifact. Each later part widens --cases to the cases it adds, so every PR in the series is gated on the GPU runner.
• TC-01 (integration generation correctness): cold request commits to the KVCR tier (local_fill), the repeat is a block-aligned hit with no failed/partial KVCR metrics.
• tests/ci/test_phase1.py: 21 CPU unit tests (telemetry parsing, token comparison, verdicts, arguments, tier configs); ci.yml now runs tests/ci next to tests/unit.

Validation

• ruff check src tests, ruff format --check, pytest tests/ci: 21 passed (run on a cluster login node, not locally).
• GPU: TC-01 passed on 2×H100 with KVCR v0.1.1 (dense Qwen3-8B and hybrid Qwen3.5-4B) as part of the full test: KVCR Phase 1 end-to-end harness (CI image build, TC-01..06, dlcluster and K8s runners) #103 run; this subset will be re-run from this branch once the image for the current main is published.

Notes

• No change to src/.
• Replaces the single large test: KVCR Phase 1 end-to-end harness (CI image build, TC-01..06, dlcluster and K8s runners) #103, which stays open for reference until the series is merged.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.