Skip to content

fix(snapshot): exit vLLM source without engine teardown - #12184

Merged
galletas1712 merged 1 commit into
mainfrom
fix/vllm-snapshot-source-exit
Jul 30, 2026
Merged

fix(snapshot): exit vLLM source without engine teardown#12184
galletas1712 merged 1 commit into
mainfrom
fix/vllm-snapshot-source-exit

Conversation

@galletas1712

@galletas1712 galletas1712 commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Summary

  • terminate the source vLLM process directly after a successful initial
    Snapshot capture;
  • avoid unwinding and destroying the checkpointed CUDA engine during source
    shutdown; and
  • leave the restored path unchanged.

Motivation

This is a generic Dynamo Snapshot/vLLM lifecycle fix, independent of GMS V1.

The initial source path raises SystemExit after successful capture. That
unwinds vLLM and may enter CUDA and EngineCore teardown after the process image
has already been checkpointed. In the observed failure, teardown produced a
source-process SIGSEGV even though capture itself had succeeded.

The successful initial-capture boundary does not need Python cleanup: the
source container is deliberately being replaced by a restore-standby
container. This change therefore exits directly with os._exit(0), matching
the existing TensorRT-LLM Snapshot lifecycle. The restored path still returns
normally.

This fix is split from #12011 because it applies to Dynamo Snapshot with or
without GMS.

Validation

Focused test

The focused unit test covers both lifecycle branches:

  • successful initial capture exits directly with status 0; and
  • restored execution returns without exiting.

nscale Snapshot E2E

The fix passed in the final graph-enabled Qwen3-0.6B TP1/DP1 Snapshot + GMS V1
proof on nscale-dev, node s2877.

The source engine logged:

Initial vLLM snapshot captured successfully; exiting without destroying the engine

Its terminal state was:

exitCode=0
reason=Completed

The replacement entered restore standby, Snapshot reported
nvidia.com/snapshot-restore-status.engine=completed, restore completed in
2.674 seconds, and the restored engine served five coherent requests with
cached_tokens=0.

The previous destructor-path SIGSEGV did not recur. No CUDA fault, illegal
memory access, late free, engine crash, or restore retry was observed.

Qualification build:
https://github.com/ai-dynamo/dynamo/actions/runs/30227329178.

Summary by CodeRabbit

  • Bug Fixes
    • Improved the snapshot capture completion flow to exit cleanly after a successful restore.
    • Added a confirmation message when the vLLM snapshot is captured successfully.

@github-actions github-actions Bot added fix backend::vllm Relates to the vllm backend labels Jul 26, 2026
@datadog-official

datadog-official Bot commented Jul 26, 2026

Copy link
Copy Markdown

Pipelines

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 40.64% (-10.28%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 58272b1 | Docs | Datadog PR Page | Give us feedback!

@galletas1712
galletas1712 force-pushed the fix/vllm-snapshot-source-exit branch from 3abe47b to 0e09c2e Compare July 27, 2026 21:52
@copy-pr-bot

copy-pr-bot Bot commented Jul 27, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@cursor
cursor Bot force-pushed the fix/vllm-snapshot-source-exit branch from 0e09c2e to 15a1614 Compare July 27, 2026 21:54
@cursor
cursor Bot force-pushed the fix/vllm-snapshot-source-exit branch from 15a1614 to 6207d3b Compare July 27, 2026 21:59
@galletas1712
galletas1712 marked this pull request as ready for review July 27, 2026 22:00
@galletas1712
galletas1712 requested review from a team as code owners July 27, 2026 22:00

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 1dc563b1-3d4f-4730-b993-1417135c0e13

📥 Commits

Reviewing files that changed from the base of the PR and between 329aba5 and 6207d3b.

📒 Files selected for processing (1)
  • components/src/dynamo/vllm/snapshot.py

Walkthrough

The vLLM snapshot success path now logs completion and terminates the process with os._exit(0) after snapshot restoration.

Changes

Snapshot restore termination

Layer / File(s) Summary
Restore completion and process exit
components/src/dynamo/vllm/snapshot.py
Imports os, logs successful snapshot capture, and replaces SystemExit(0) with os._exit(0) after restore completion.

Estimated code review effort: 2 (Simple) | ~5 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description covers summary, motivation, and validation, but it misses the required reviewer-start section and the required issue-linking section. Add the required template sections: Overview, Details, Where should reviewer start?, and Related Issues with either a Closes/Relates reference or the no-issue checkbox.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly matches the main change: vLLM source now exits directly after snapshot capture to avoid engine teardown.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@galletas1712

Copy link
Copy Markdown
Contributor Author

/ok to test 6207d3b

@galletas1712
galletas1712 enabled auto-merge (squash) July 27, 2026 23:27
@galletas1712
galletas1712 force-pushed the fix/vllm-snapshot-source-exit branch from 6207d3b to f02b4cb Compare July 28, 2026 03:48
@galletas1712

Copy link
Copy Markdown
Contributor Author

/ok to test f02b4cb

@galletas1712
galletas1712 force-pushed the fix/vllm-snapshot-source-exit branch 2 times, most recently from 4232724 to 7b55ac9 Compare July 28, 2026 21:12
@galletas1712

Copy link
Copy Markdown
Contributor Author

/ok to test 7b55ac9

@galletas1712
galletas1712 force-pushed the fix/vllm-snapshot-source-exit branch from 7b55ac9 to d8a06c3 Compare July 29, 2026 20:02
@galletas1712

Copy link
Copy Markdown
Contributor Author

/ok to test d8a06c3

@galletas1712
galletas1712 force-pushed the fix/vllm-snapshot-source-exit branch from d8a06c3 to b1f88b2 Compare July 30, 2026 00:45
@galletas1712

Copy link
Copy Markdown
Contributor Author

/ok to test b1f88b2

@galletas1712
galletas1712 force-pushed the fix/vllm-snapshot-source-exit branch from b1f88b2 to afc98d1 Compare July 30, 2026 06:43
@galletas1712
galletas1712 requested review from a team as code owners July 30, 2026 06:43
@galletas1712

Copy link
Copy Markdown
Contributor Author

/ok to test afc98d1

After a successful initial capture, exit the source process immediately
with os._exit so engine subprocess teardown cannot keep the checkpoint
Job alive or SIGSEGV during CUDA cleanup.

Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
@galletas1712

Copy link
Copy Markdown
Contributor Author

/ok to test 58272b1

@galletas1712
galletas1712 force-pushed the fix/vllm-snapshot-source-exit branch from afc98d1 to 58272b1 Compare July 30, 2026 16:51
@galletas1712
galletas1712 requested review from a team and removed request for a team July 30, 2026 16:51
@galletas1712
galletas1712 merged commit be65a21 into main Jul 30, 2026
174 of 176 checks passed
@galletas1712
galletas1712 deleted the fix/vllm-snapshot-source-exit branch July 30, 2026 19:04
nv-nmailhot pushed a commit that referenced this pull request Aug 3, 2026
…12184) (#12490)

Signed-off-by: Schwinn Saereesitthipitak <galletas1712@users.noreply.github.com>
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants