[Llama.py -> mistral.py] Extract mistral-only relevant code into separate file

[patrickvonplaten]

We're adding more and more mistral-only code to the llama.py class which makes it harder to read and creates possible future unwanted dependencies. E.g. if other models depend on the llama.py class one might think that mistral-only code might also be relevant for such classes and thus make vLLM too rigid.

[patrickvonplaten]

this code comes from a very old PR: #8168 and I'm quite convinced that it's only mistral checkpoints that actually make use of this function so moving it out

[patrickvonplaten]

only mistral makes use of llama_4 scaling

[patrickvonplaten]

afaik only mistral-nemo every used this

[patrickvonplaten]

Doesn't seem to be the case 😅

[mergify]

Hi @patrickvonplaten, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[gemini-code-assist]

Code Review

The pull request successfully extracts Mistral-specific model adaptations into a new file, mistral.py, and refactors llama.py to be more generic. This improves modularity and maintainability of the codebase. The changes in llama.py and registry.py are appropriate for this refactoring.

I am having trouble creating individual review comments. Click here to see my feedback.

vllm/model_executor/mistral.py (214-233)

The logic within maybe_remap_mistral for handling wq and wk weights, especially with the conditional checks for qscale_weight and loaded_weight.numel() > 1, is quite complex and repetitive. This intricate logic increases the potential for errors and makes future modifications or debugging challenging. Consider refactoring this section to improve clarity and reduce duplication, perhaps by abstracting the common permutation and conditional checks into smaller, more focused helper functions.

[DarkLight1337]

Thanks for the cleanup!

[patrickvonplaten]

Hmm the docker building is probably flaky no @DarkLight1337 ?

[patrickvonplaten]

Think final failing tests are unrelated:

[2026-01-21T19:36:47Z] ERROR entrypoints/openai/test_serving_chat.py::TestGPTOSSChat::test_gpt_oss_chat_tool_call_streaming[with_tool_parser-exclude_tools_when_tool_choice_none] - RuntimeError: Server failed to start in time.
[2026-01-21T18:50:16Z] ERROR entrypoints/openai/responses/test_parsable_context.py::test_basic[Qwen/Qwen3-8B] - RuntimeError: Server failed to start in time.

good to merge you think @DarkLight1337 ?