[Attention] Refactor check_and_update_config

[MatthewBonanni]

Purpose

check_and_update_config unnecessarily duplicates much of the logic from the attention selector in order to set an approproate block size. This PR refactors check_and_update_config to use the selector, which will be simpler to maintain going forward.

• If the user specifies --block-size and --attention-backend and the backend doesn't support the block size, we raise an error rather than overriding a user selection
• If the user specifies --block-size but not the backend, the backend selector respects that block size choice and tries to find a backend which is compatible, raising an error if no valid backends are found
  • If the user-selected block size forces the selection of a non-optimal backend, the user is warned about this
• If the user specifies --attention-backend only, an appropriate block size is selected
• If the user specifies neither, an appropriate attention backend is selected, and an appropriate block size for that backend is selected

Test Plan

Automatic selection

vllm serve deepseek-ai/DeepSeek-V2-Lite-Chat --attention-backend=FLASHMLA

yields

Setting kv cache block size to 64 for FLASHMLA backend.

Setting bad block size

vllm serve deepseek-ai/DeepSeek-V2-Lite-Chat --attention-backend=FLASHMLA --block-size 32

yields

(APIServer pid=1710084) pydantic_core._pydantic_core.ValidationError: 1 validation error for VllmConfig
(APIServer pid=1710084)   Value error, User-specified block_size=16 is incompatible with FLASH_ATTN_MLA backend (requires block_size=128). Either remove --block-size to auto-select, or use --block-size 128. [type=value_error, input_value=ArgsKwargs((), {'model_co...transfer_config': None}), input_type=ArgsKwargs]

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @MatthewBonanni.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[gemini-code-assist]

Code Review

This pull request refactors the attention backend selection logic in check_and_update_config and get_attn_backend_cls. The changes significantly improve code clarity and maintainability by centralizing the backend selection logic into a new select_attention_backend method and introducing get_preferred_block_size for determining block sizes. This is a great improvement. I've found one issue with a type hint that should be addressed.

[mergify]

Hi @MatthewBonanni, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[pavanimajety]

Thanks for the PR, @MatthewBonanni!
Looks good to me, pending clean CI and minor nits!

[mergify]

Hi @MatthewBonanni, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[mgoin]

Great work, LGTM!

[DarkLight1337]

I think this PR broke model initialization and V1 Others tests, checking in https://buildkite.com/vllm/ci/builds/52009/steps/canvas

The build for the previous PR for comparison: https://buildkite.com/vllm/ci/builds/52001/steps/canvas

[njhill]

Do we understand how this was merged with so many breakages? Have we given up on the rule that we don't force merge without certainty that CI failures are unrelated (even if it seems obvious that they are)? Has @vllm-bot gone rogue?

[mgoin]

I went rogue, sorry. I looked at the failures but thought they were not related

[MatthewBonanni]

My fault too, sorry. I asked @mgoin yesterday morning whether we should consider a force merge on this. I also assumed failures were unrelated.