[megatron] feat: LoRA adapter only refit (TensorLoRARequest) by HollowMan6 · Pull Request #4632 · verl-project/verl

HollowMan6 · 2025-12-22T01:01:51Z

What does this PR do?

Checklist Before Starting

Search for similar PRs. Paste at least one query link here: ...
Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
- {modules} include fsdp, megatron, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data, cfg, reward
- If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
- {type} is in feat, fix, refactor, chore, test
- If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
- Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc.

API and Usage Example

Demonstrate how the API changes if any, and provide usage example(s) if possible.

# Add code snippet or script demonstrating how to use this

Design & Code Changes

Demonstrate the high-level design if this PR is complex, and list the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

Read the Contribute Guide.
Apply pre-commit checks: pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always
Add / Update the documentation.
Add unit or end-to-end test(s) to the CI workflow to cover all the code. If not feasible, explain why: ...
Once your PR is ready for CI, send a message in the ci-request channel in the verl Slack workspace. (If not accessible, please try the Feishu group (飞书群).)

gemini-code-assist

Code Review

This pull request introduces a significant feature for LoRA adapter-only weight updates in the Megatron backend, which avoids the overhead of merging adapters into the base model for each weight synchronization. The changes are well-structured and include updates to documentation, configuration handling for LoRA, new PEFT utility functions for vLLM compatibility, and refactored weight export logic. The implementation appears solid and aligns with the stated goals. I have a couple of suggestions to enhance code maintainability and clarity regarding duplicated logic and a confusing condition.

verl/utils/config.py

verl/workers/megatron_workers.py

gemini-code-assist

Code Review

This pull request introduces a valuable feature for LoRA adapter-only refitting in the Megatron backend, which should provide significant performance benefits. The changes across documentation, configuration, and worker implementations appear to correctly support both merging LoRA adapters and loading them separately. My primary concern, detailed in a specific comment, is the repeated implementation of LoRA configuration logic across several files. Addressing this will improve the long-term maintainability of the codebase.

verl/workers/rollout/vllm_rollout/vllm_rollout.py

gemini-code-assist

Code Review

This pull request introduces a significant feature for LoRA adapter-only refit with the Megatron backend, which is a great addition for performance and flexibility. The changes are extensive, touching documentation, examples, configuration, and core worker logic. The implementation of separate synchronization for base and adapter weights is well-thought-out.

I've identified a few critical logic issues in the configuration handling for LoRA, particularly concerning the new merge option and backward compatibility with older LoRA settings. These issues are present in three different files, indicating that the logic is duplicated. For better maintainability, I recommend refactoring this configuration logic into a shared utility function to ensure consistency and make future updates easier. Addressing these points will significantly improve the robustness of this new feature.

verl/utils/config.py

verl/workers/rollout/vllm_rollout/vllm_async_server.py

verl/workers/rollout/vllm_rollout/vllm_rollout.py

verl/utils/config.py

ISEEKYAN · 2026-01-22T11:50:11Z

verl/utils/megatron_peft_utils.py


 import torch

+# Map megatron lora target modules to HF-style module names for vLLM


This is only an advice, not necessary, can we move the lora mapping to megatron-bridge instead of keeping them in verl?

Yes, I think this is something in discussion, and I guess it would need some API designs.

add @yaoyu-33 for vis. We need to extend bridge APIs with capability of exporting lora weights.

verl/workers/megatron_workers.py

HollowMan6 · 2026-01-23T12:10:20Z

cc: @wuxibin89 @vermouth1992 for further comments

Signed-off-by: Hollow Man <hollowman@opensuse.org>

…oject#4632) ### What does this PR do? <img width="2206" height="1314" alt="lora-performance" src="https://github.com/user-attachments/assets/0482f423-01a3-4e52-a7ee-8b9cd79b7b1a" /> <img width="2208" height="1800" alt="lora-critic-val-score" src="https://github.com/user-attachments/assets/6ce10400-8164-47d8-90a6-c1bf002fb9e8" /> <img width="2204" height="1794" alt="lora-actor-plus-rollout-mismatch" src="https://github.com/user-attachments/assets/092d3a43-4eba-425e-a584-8d83c1f02de4" /> ### Checklist Before Starting - [X] Search for similar PRs. Paste at least one query link here: ... - [X] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) - `{modules}` include `fsdp`, `megatron`, `sglang`, `vllm`, `rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`, `deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`, `model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data`, `cfg`, `reward` - If this PR involves multiple modules, separate them with `,` like `[megatron, fsdp, doc]` - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test` - If this PR breaks any API (CLI arguments, config, function signature, etc.), add `[BREAKING]` to the beginning of the title. - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching` ### Test > For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc. ### API and Usage Example > Demonstrate how the API changes if any, and provide usage example(s) if possible. ```python # Add code snippet or script demonstrating how to use this ``` ### Design & Code Changes > Demonstrate the high-level design if this PR is complex, and list the specific changes. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [X] Read the [Contribute Guide](https://github.com/volcengine/verl/blob/main/CONTRIBUTING.md). - [X] Apply [pre-commit checks](https://github.com/volcengine/verl/blob/main/CONTRIBUTING.md#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [X] Add / Update [the documentation](https://github.com/volcengine/verl/tree/main/docs). - [X] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/volcengine/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [X] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ). (If not accessible, please try [the Feishu group (飞书群)](https://applink.larkoffice.com/client/chat/chatter/add_by_link?link_token=772jd4f1-cd91-441e-a820-498c6614126a).) Signed-off-by: Hollow Man <hollowman@opensuse.org>

HollowMan6 changed the title ~~[megatron] feat: LoRA adapter only weight update~~ [megatron] feat: LoRA adapter only weight update (TensorLoRARequest) Dec 22, 2025

HollowMan6 changed the title ~~[megatron] feat: LoRA adapter only weight update (TensorLoRARequest)~~ [megatron] feat: LoRA adapter only refit (TensorLoRARequest) Dec 22, 2025

gemini-code-assist bot reviewed Dec 22, 2025

View reviewed changes

verl/utils/config.py Outdated Show resolved Hide resolved

verl/workers/megatron_workers.py Show resolved Hide resolved

HollowMan6 force-pushed the lora_adapters_update branch 4 times, most recently from f6706bc to 81d261e Compare December 22, 2025 01:33

HollowMan6 mentioned this pull request Dec 22, 2025

[BugFix] LoRA: Support loading base_layer of experts vllm-project/vllm#31104

Merged

5 tasks

HollowMan6 force-pushed the lora_adapters_update branch 5 times, most recently from 23b8716 to 797dc7f Compare December 28, 2025 14:01

HollowMan6 force-pushed the lora_adapters_update branch 2 times, most recently from dddcffe to 91d4732 Compare December 31, 2025 08:43

gemini-code-assist bot reviewed Dec 31, 2025

View reviewed changes

verl/workers/rollout/vllm_rollout/vllm_rollout.py Outdated Show resolved Hide resolved

HollowMan6 force-pushed the lora_adapters_update branch 3 times, most recently from 1ad0f96 to 0c7b07a Compare January 7, 2026 01:51

HollowMan6 force-pushed the lora_adapters_update branch 4 times, most recently from 49a4289 to cb4bfe1 Compare January 9, 2026 23:23

HollowMan6 mentioned this pull request Jan 9, 2026

[megatron] feat: Share actor and ref in LoRA #4673

Merged

7 tasks

HollowMan6 force-pushed the lora_adapters_update branch 2 times, most recently from ef83292 to 8254e91 Compare January 12, 2026 23:15

HollowMan6 marked this pull request as ready for review January 12, 2026 23:16

Copilot AI review requested due to automatic review settings January 12, 2026 23:16

HollowMan6 requested review from ISEEKYAN and vermouth1992 as code owners January 12, 2026 23:16

gemini-code-assist bot reviewed Jan 13, 2026

View reviewed changes

verl/utils/config.py Show resolved Hide resolved

verl/workers/rollout/vllm_rollout/vllm_async_server.py Show resolved Hide resolved

verl/workers/rollout/vllm_rollout/vllm_rollout.py Outdated Show resolved Hide resolved

HollowMan6 force-pushed the lora_adapters_update branch 10 times, most recently from a7aecaf to ce759a7 Compare January 20, 2026 19:08

HollowMan6 force-pushed the lora_adapters_update branch from ce759a7 to 7ba73c0 Compare January 21, 2026 15:02

ISEEKYAN reviewed Jan 22, 2026

View reviewed changes

HollowMan6 requested a review from ISEEKYAN January 22, 2026 12:36

ISEEKYAN approved these changes Jan 22, 2026

View reviewed changes

HollowMan6 force-pushed the lora_adapters_update branch from 2629b6a to 7ad4c1d Compare January 23, 2026 14:08

HollowMan6 requested a review from tardis-key as a code owner January 23, 2026 14:08

HollowMan6 force-pushed the lora_adapters_update branch 2 times, most recently from 43487b6 to c49313f Compare January 23, 2026 15:59

[megatron] feat: LoRA adapter only refit (TensorLoRARequest)

30a0782

Signed-off-by: Hollow Man <hollowman@opensuse.org>

HollowMan6 force-pushed the lora_adapters_update branch from c49313f to 30a0782 Compare January 23, 2026 19:58

ISEEKYAN approved these changes Jan 24, 2026

View reviewed changes

ISEEKYAN merged commit b703663 into verl-project:main Jan 24, 2026
83 of 103 checks passed

HollowMan6 deleted the lora_adapters_update branch January 24, 2026 07:21

HollowMan6 mentioned this pull request Feb 4, 2026

[fsdp] feat: Merge lora in fsdp training to speed up rollout #5115

Merged

6 tasks

erictang000 mentioned this pull request Mar 17, 2026

[train] Support LoRA-only weight syncing for Megatron backend NovaSky-AI/SkyRL#1336

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[megatron] feat: LoRA adapter only refit (TensorLoRARequest)#4632

[megatron] feat: LoRA adapter only refit (TensorLoRARequest)#4632
ISEEKYAN merged 1 commit intoverl-project:mainfrom
HollowMan6:lora_adapters_update

HollowMan6 commented Dec 22, 2025 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ISEEKYAN Jan 22, 2026

Uh oh!

HollowMan6 Jan 22, 2026

Uh oh!

ISEEKYAN Jan 22, 2026

Uh oh!

Uh oh!

HollowMan6 commented Jan 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		import torch

		# Map megatron lora target modules to HF-style module names for vLLM

Conversation

HollowMan6 commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist Before Starting

Test

API and Usage Example

Design & Code Changes

Checklist Before Submitting

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ISEEKYAN Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

HollowMan6 Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

ISEEKYAN Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

HollowMan6 commented Jan 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HollowMan6 commented Dec 22, 2025 •

edited

Loading