Skip to content

fix(vllm): keep prefix reuse separate from P-to-D handoff ownership [D29] - #18047

Closed
yifjiang wants to merge 1 commit into
codex/kv-reuse-d28-gpu-capability-publicationfrom
codex/kv-reuse-d29-prefix-handoff-ownership
Closed

yifjiang wants to merge 1 commit into
codex/kv-reuse-d28-gpu-capability-publicationfrom
codex/kv-reuse-d29-prefix-handoff-ownership

Conversation

@yifjiang

@yifjiang yifjiang commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Prefill prefix loading and post-prefill decode handoff remain separate; preserve child-engine IDs, metadata and NIXL/Mooncake protocol shape.

Review slice D29 of #16748; part of #16747 and #16746. This draft targets codex/kv-reuse-d28-gpu-capability-publication for a complete incremental review diff of 291 additions plus deletions, including tests, documentation and extraction glue.

Reuse remains experimental and default off. Retained inventory is advisory, complete compatible rank parts must precede readiness, and native operations retain ownership until quiescence. Supported GPU activation also requires the companion KVCR/native slices and the declared deployment profile.

Validation

All configured pre-commit hooks passed for this exact slice on allocated SLURM. Stack-level CPU regression coverage is summarized on D31; each draft still requires its own applicable CI in the Checks tab. Historical combined-head and hardware results retain their original source scope.

@github-actions github-actions Bot added fix backend::vllm Relates to the vllm backend labels Nov 4, 2026
@devin-ai-integration

devin-ai-integration Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

❌ Dynamo PR CI failed — run 41941714643 (attempt 1) on 47ca56251a

Gate checks: ❌ backend-status-check · ✅ deploy-status-check · ✅ dynamo-status-check

Framework 1-GPU arm64
vLLM ❌ 1

Failure details

1 job failed: both vLLM arm64 CPU pytest stages died in the self-hosted runner's container hook before pytest printed any test results. This looks like infrastructure, not a test failure.

❌ vllm-runtime / Test cuda13.0, arm64: custom container implementation failed (runner)

Job: vllm-runtime / Test cuda13.0, arm64 · Failed step: Run CPU-only tests (parallelized) · Logs: gh run view --job 126985293053 -R ai-dynamo/dynamo --log-failed

▶️ Executing (cwd=/workspace): python -m pytest -n auto --dist=loadscope --continue-on-collection-errors -v ...
##[error]Executing the custom container implementation failed. Please contact your self hosted runner administrator.
...
▶️ Executing (cwd=/workspace): python -m pytest -n 0 --dist=loadscope --continue-on-collection-errors -v ...
##[error]Executing the custom container implementation failed. Please contact your self hosted runner administrator.
No files were found with the provided path: test-results/allure-results/. No artifacts will be uploaded.
❌ pre_merge_cpu: step outcome 'failure'
❌ pre_merge_cpu_e2e: step outcome 'failure'
##[error]A pytest stage did not record a passing result. See this step's log for the stage.

The sanity check passed. Both the pre_merge_cpu stage (xdist, -n auto) and the pre_merge_cpu_e2e stage (-n 0) were cut off by the runner's k8s container hook (Executing the custom container implementation failed) after only Datadog plugin startup. No test was reported as FAILED, and no JUnit or Allure results were written. This is an infrastructure signature on the arm64 self-hosted runner, so rerunning the failed jobs is likely enough.

For agents
{"pr": 18047, "run_id": 41941714643, "run_attempt": 1, "head_sha": "47ca56251a3087a49c1b78c49de5304c221598c7", "failures": [{"job": "vllm-runtime / Test cuda13.0, arm64", "job_id": 126985293053, "failed_step": "Run CPU-only tests (parallelized)", "signature": "runner error: Executing the custom container implementation failed (no pytest output)", "tests": [], "log_cmd": "gh run view --job 126985293053 -R ai-dynamo/dynamo --log-failed"}]}

Posted automatically by Devin for run 41941714643. Updated on every full-CI run of this PR.

@yifjiang
yifjiang force-pushed the codex/kv-reuse-d28-gpu-capability-publication branch from 373a860 to f741153 Compare November 4, 2026 08:16
@yifjiang
yifjiang force-pushed the codex/kv-reuse-d29-prefix-handoff-ownership branch 2 times, most recently from c63273d to 9d80e26 Compare November 4, 2026 08:28
@yifjiang
yifjiang force-pushed the codex/kv-reuse-d28-gpu-capability-publication branch from f741153 to b8fa9ba Compare November 4, 2026 08:28
@yifjiang
yifjiang force-pushed the codex/kv-reuse-d29-prefix-handoff-ownership branch from 9d80e26 to c9ab21c Compare November 4, 2026 08:34
@yifjiang
yifjiang force-pushed the codex/kv-reuse-d28-gpu-capability-publication branch 2 times, most recently from 7ef9e4a to 0b2f6e6 Compare November 4, 2026 08:54
@yifjiang
yifjiang force-pushed the codex/kv-reuse-d29-prefix-handoff-ownership branch 2 times, most recently from 0205bc6 to 368bd89 Compare November 4, 2026 20:39
@yifjiang
yifjiang force-pushed the codex/kv-reuse-d28-gpu-capability-publication branch from 0b2f6e6 to 1cd11a0 Compare November 4, 2026 20:39
@copy-pr-bot

copy-pr-bot Bot commented Nov 4, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

1 similar comment
@copy-pr-bot

copy-pr-bot Bot commented Nov 4, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

yifjiang commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 368bd89

Signed-off-by: Yifan Jiang <19356972+yifjiang@users.noreply.github.com>
@yifjiang
yifjiang force-pushed the codex/kv-reuse-d29-prefix-handoff-ownership branch from 368bd89 to 47ca562 Compare November 5, 2026 00:24
@yifjiang
yifjiang force-pushed the codex/kv-reuse-d28-gpu-capability-publication branch from 1cd11a0 to 02d492b Compare November 5, 2026 00:24
@yifjiang

yifjiang commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 47ca562

@yifjiang yifjiang closed this Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend::vllm Relates to the vllm backend fix size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant