[Bugfix] Invalidate startup plans on GPU budget changes

[spa5k]

Overview

A startup plan can reuse too much KV memory after gpu_memory_utilization is reduced. This change includes that setting in the memory plan key, so a changed budget gets a new profile.

Claims

• Reject a saved plan when GPU memory utilization changes.
• Reuse the plan when the setting stays the same.
• Preserve compiler-cache reuse and the priority of an explicit KV budget.

Validation

Check	Baseline	Patch
Current-base startup-plan cases	2 failed	2 passed
Worker suite in recorded native validation	—	22 passed
KV allowance after utilization falls from 0.8 to 0.35	Reused 16.15 GiB	Profiled 6.23 GiB

The GPU test used Qwen3-0.6B BF16 on one L4. The new total GPU budget was only 7.71 GiB, so the reused 16.15 GiB allowance exceeded it. Five eager starts returned ready. Five compiled starts also checked invalidation and same-budget reuse; all 20 chat requests matched prompt and output token IDs.

An explicit 2 GiB budget bypassed plan use and save. A fixed-budget control matched all 16 requests across four starts. One comparison across different KV budgets changed a text continuation. These are serving checks, rather than a general model accuracy result. No startup or request-speed gain is claimed.

The current-base startup-plan regression checks passed. Local pre-commit and mypy checks for Python 3.10 and 3.12 passed. Upstream test CI is blocked by the contributor gate.

Details

The compiler hash excludes GPU memory utilization. The memory plan needs this separate factor because its saved KV allowance depends on the budget. The patch adds that factor to the existing fingerprint.

Old plans require one new memory profile. Device, rank, version, and free-memory checks still apply. The plan format is unchanged.

Test commands and environment

The repository worker test file was copied unchanged to /work/refresh-source/60143/test.py for the native checks. Both the current-base and candidate Python modules were installed in turn.

/opt/audit/.venv/bin/python -m pytest /work/refresh-source/60143/test.py --confcutdir=/work/refresh-source/60143 -q -k startup_plan
pre-commit run --files vllm/v1/worker/startup_plan.py tests/v1/worker/test_gpu_worker.py
pre-commit run mypy-3.12 --files vllm/v1/worker/startup_plan.py tests/v1/worker/test_gpu_worker.py --hook-stage manual

The broader worker-suite counts above are from the recorded matching-wheel checks. A newer-wheel full-suite run produced 22 test progress markers but timed out before pytest completed; it does not count as a pass. The current-base refresh uses the focused startup-plan suite.

Validation used a prebuilt Linux vLLM wheel with Python source overlays. The full source tree was not compiled. No compiled code changed.

Live tests used Qwen/Qwen3-0.6B, BF16, one L4, and Python 3.12. Compiled tests used MRV2, context length 4096, batch token limit 2048, sequence limit 16, and seed 42. Their lower-budget KV profile was 6.09 GiB. The eager test used context length 16384. A 256 MiB allocation during the first start was released before reuse to satisfy the existing free-memory gate.

To reproduce, enable VLLM_ENABLE_STARTUP_PLAN=1 and use the same cache directory for starts at utilization 0.8 and 0.35. The second start must have at least the free memory recorded by the first start. The patch profiles again at the lower budget. A later start with that budget can reuse its matching plan.

AI assistance was used.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment /ci run for upstream CI or /amd-ci run for AMD CI only whenever CI signals are needed.

Once the PR is approved or has the ready label, the PR author can also use the corresponding /ci run, /ci retry, and /ci cancel commands, or their /amd-ci variants. New commits do not start upstream CI automatically.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.