Remove unused swap_space parameter#27988
Conversation
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run You ask your reviewers to trigger select CI tests on top of Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. 🚀 |
|
Documentation preview: https://vllm--27988.org.readthedocs.build/en/27988/ |
There was a problem hiding this comment.
Code Review
This pull request aims to remove the unused swap_space parameter throughout the codebase. The changes are comprehensive, touching documentation, tests, and core configuration files. While most of the changes are correct, I've identified a critical issue in vllm/engine/arg_utils.py that appears to be a typo. This typo will likely cause a KeyError during command-line argument parsing, breaking the CLI. I've provided a suggestion to fix this.
vllm/engine/arg_utils.py
Outdated
| @@ -880,8 +879,7 @@ def add_cli_args(parser: FlexibleArgumentParser) -> FlexibleArgumentParser: | |||
| cache_group.add_argument( | |||
| "--kv-cache-memory-bytes", **cache_kwargs["kv_cache_memory_bytes"] | |||
| ) | |||
| cache_group.add_argument("--swap-space", **cache_kwargs["swap_space"]) | |||
| cache_group.add_argument("--kv-cache-dtype", **cache_kwargs["cache_dtype"]) | |||
| cache_group.add_argument("--kv-cache-dtype", **cache_kwargs["kv_cache_dtype"]) | |||
There was a problem hiding this comment.
There seems to be a typo here. The get_kwargs(CacheConfig) function returns a dictionary with keys corresponding to the fields in CacheConfig. CacheConfig has a field named cache_dtype, but not kv_cache_dtype. Accessing cache_kwargs["kv_cache_dtype"] will likely cause a KeyError. This should probably be cache_kwargs["cache_dtype"] as it was before this change.
| cache_group.add_argument("--kv-cache-dtype", **cache_kwargs["kv_cache_dtype"]) | |
| cache_group.add_argument("--kv-cache-dtype", **cache_kwargs["cache_dtype"]) |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
vllm/engine/arg_utils.py
Outdated
| cache_group.add_argument( | ||
| "--kv-cache-memory-bytes", **cache_kwargs["kv_cache_memory_bytes"] | ||
| ) | ||
| cache_group.add_argument("--swap-space", **cache_kwargs["swap_space"]) | ||
| cache_group.add_argument("--kv-cache-dtype", **cache_kwargs["cache_dtype"]) | ||
| cache_group.add_argument("--kv-cache-dtype", **cache_kwargs["kv_cache_dtype"]) |
There was a problem hiding this comment.
Fix CLI lookup for kv cache dtype option
The new code builds the CLI by fetching cache_kwargs["kv_cache_dtype"], but the CacheConfig dataclass still exposes the field as cache_dtype. get_kwargs(CacheConfig) therefore produces a dictionary with key "cache_dtype", and this lookup now raises a KeyError as soon as the argument parser is constructed (e.g. running python -m vllm.entrypoints.llm --help). This prevents the CLI from starting at all. Keep the lookup keyed on "cache_dtype" or rename the field consistently.
Useful? React with 👍 / 👎.
85c7aaf to
2d12c02
Compare
|
Hey @hmellor , we don't use that at all. |
|
Hi @mcelrath, the pre-commit checks have failed. Please run: uv pip install pre-commit
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, |
|
This pull request has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this pull request should remain open. Thank you! |

References: #27984
Purpose
Remove unused
swap_spaceparameter after the removal ofbest_ofsampling. Fixes #27984Test Plan
(Untested) working on it.
Help would be appreciated. I'm having quite a lot of difficulty compiling vllm and all its dependencies. This should be a simple fix with no impact anywhere else.
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.