Skip to content

[MRV2] Add shutdown() method#41297

Merged
WoosukKwon merged 2 commits into
mainfrom
woosuk/mrv2-shutdown
May 3, 2026
Merged

[MRV2] Add shutdown() method#41297
WoosukKwon merged 2 commits into
mainfrom
woosuk/mrv2-shutdown

[MRV2] Add shutdown() method

73e2316
Select commit
Loading
Failed to load commit list.
Claude / Claude Code Review completed Apr 29, 2026 in 6m 8s

Code review found 1 important issue

Found 5 candidates, confirmed 3. See review comments for details.

Details

Severity Count
🔴 Important 1
🟡 Nit 0
🟣 Pre-existing 0
Severity File:Line Issue
🔴 Important vllm/v1/worker/gpu/model_runner.py:1340-1356 shutdown() does not actually release model weights, CUDA graphs, or auxiliary GPU buffers

Annotations

Check failure on line 1356 in vllm/v1/worker/gpu/model_runner.py

See this annotation in the file changed.

@claude claude / Claude Code Review

shutdown() does not actually release model weights, CUDA graphs, or auxiliary GPU buffers

The new `shutdown()` does not actually release the bulk of GPU memory it claims to free: `self.model_state`, `self.speculator`, `self.pooling_runner`, and `self.cudagraph_manager` all hold strong references to the model weights and/or large GPU buffers (captured CUDA graphs are typically several GiB), so `del self.model` followed by `gc.collect()`/`empty_cache()` cannot reclaim them. Also missing are `self.intermediate_tensors` (sized at `max_num_batched_tokens` on non-first PP ranks) and `self.