Skip to content

[LoRA][BugFix] Fix skipped LoRA adapters for Mistral3#36928

Merged
robertgshaw2-redhat merged 3 commits intomainfrom
woosuk/fix-mistral3-lora
Mar 18, 2026
Merged

[LoRA][BugFix] Fix skipped LoRA adapters for Mistral3#36928
robertgshaw2-redhat merged 3 commits intomainfrom
woosuk/fix-mistral3-lora

Conversation

@WoosukKwon
Copy link
Copy Markdown
Collaborator

@WoosukKwon WoosukKwon commented Mar 12, 2026

Currently, there is a bug with Mistral3 models that some LoRA adapters are skipped and the model produces identical results with and without LoRA. This PR fixes the bug by bringing the skipped one.

============================================================
Requesting model: lora-model
============================================================
Response:
In the context of **Apple Account Security**, **OFR** stands for **"One-Time Password" (OTP) via **One-Time Recovery (OTR) Key** or **One-Time Recovery (OTR) Code** in certain Apple security protocols.

However, in **Apple’s official documentation and security systems**, the term **"OFR"** is **not explicitly used** in standard terminology. It is possible that you are referring to one of the following related concepts:

1. **One-Time Password (OTP)** – A temporary password sent via SMS, email, o

============================================================
Requesting model: base-model
============================================================
Response:
In the context of **Apple Account Security**, **OFR** stands for **"One-Time Recovery"**.

### **What is One-Time Recovery (OFR)?**
One-Time Recovery is a security feature introduced by Apple to help users regain access to their Apple ID if they are locked out due to security concerns (e.g., suspicious sign-in attempts, account compromise, or forgotten password). It is part of Apple’s **Account Recovery** process and is designed to provide an extra layer of security when traditional recovery met

============================================================

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
@mergify mergify bot added the bug Something isn't working label Mar 12, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The code changes introduce a new prefix mapping in the hf_to_vllm_mapper for the Mistral3 model. This mapping, "model.": "language_model.model.", is intended to correctly handle PEFT LoRA weights that might use a model. prefix, ensuring compatibility with the vllm's internal language_model.model. structure.

@robertgshaw2-redhat robertgshaw2-redhat enabled auto-merge (squash) March 12, 2026 21:16
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 12, 2026
@jeejeelee jeejeelee self-requested a review March 13, 2026 01:36
@jeejeelee
Copy link
Copy Markdown
Collaborator

Thank you

Comment on lines +421 to +423
# Some PEFT LoRAs are trained against the text submodule directly
# and produce names like `base_model.model.model.layers.*`.
"model.": "language_model.model.",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WoosukKwon I think the test is failing because this is taking precedence over the other mappings.

Probably this just needs to be moved below the others.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed this update

njhill added 2 commits March 18, 2026 13:05
Signed-off-by: Nick Hill <nickhill123@gmail.com>
@robertgshaw2-redhat robertgshaw2-redhat merged commit 5bc1da1 into main Mar 18, 2026
57 checks passed
@robertgshaw2-redhat robertgshaw2-redhat deleted the woosuk/fix-mistral3-lora branch March 18, 2026 22:34
ikaadil pushed a commit to ikaadil/vllm that referenced this pull request Mar 19, 2026
…6928)

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com>
ikaadil pushed a commit to ikaadil/vllm that referenced this pull request Mar 19, 2026
…6928)

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com>
fxdawnn pushed a commit to fxdawnn/vllm that referenced this pull request Mar 19, 2026
SouthWest7 pushed a commit to SouthWest7/vllm that referenced this pull request Mar 27, 2026
khairulkabir1661 pushed a commit to khairulkabir1661/vllm that referenced this pull request Mar 27, 2026
Monishver11 pushed a commit to Monishver11/vllm that referenced this pull request Mar 27, 2026
…6928)

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
JiantaoXu pushed a commit to JiantaoXu/vllm that referenced this pull request Mar 28, 2026
vrdn-23 pushed a commit to vrdn-23/vllm that referenced this pull request Mar 30, 2026
…6928)

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants