[MRV2] Add shutdown() method#41297
Merged
Merged
Claude / Claude Code Review
completed
Apr 29, 2026 in 6m 8s
Code review found 1 important issue
Found 5 candidates, confirmed 3. See review comments for details.
Details
| Severity | Count |
|---|---|
| 🔴 Important | 1 |
| 🟡 Nit | 0 |
| 🟣 Pre-existing | 0 |
| Severity | File:Line | Issue |
|---|---|---|
| 🔴 Important | vllm/v1/worker/gpu/model_runner.py:1340-1356 |
shutdown() does not actually release model weights, CUDA graphs, or auxiliary GPU buffers |
Annotations
Check failure on line 1356 in vllm/v1/worker/gpu/model_runner.py
claude / Claude Code Review
shutdown() does not actually release model weights, CUDA graphs, or auxiliary GPU buffers
The new `shutdown()` does not actually release the bulk of GPU memory it claims to free: `self.model_state`, `self.speculator`, `self.pooling_runner`, and `self.cudagraph_manager` all hold strong references to the model weights and/or large GPU buffers (captured CUDA graphs are typically several GiB), so `del self.model` followed by `gc.collect()`/`empty_cache()` cannot reclaim them. Also missing are `self.intermediate_tensors` (sized at `max_num_batched_tokens` on non-first PP ranks) and `self.
Loading