[LoRA] Fix LoRA merge and support CanonicalLoRA merge

[HollowMan6]

What does this PR do ?

Previous LoRA merge #1310 contains several errors, this PR aims to fix it and support CanonicalLoRA merge

Changelog

• Handle fused QKV/gate up correctly
• Handle PP gathering correctly

GitHub Actions CI

See the CI sectionin the Contributing doc for how to trigger the CI. A Nvidia developer will need to approve and trigger the CI for external contributors.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

If you haven't finished some of the above items you can still open "Draft" PR.

Additional Information

• Related to feat: Support PEFT weight mapping and merge LoRA adapters when export to hf #1310

_{✨ Presented to you with Mind Lab - A Lab for Experiential Intelligence.}

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[HollowMan6]

cc: @yaoyu-33 @cuichenx @ananthsub

verl-project/verl#4063 (comment)

[HollowMan6]

Now this PR is refactored for better support of future LoRA bridge, we first construct the AdapterWeightConversionTask using the information from _megatron_global_adapters_info_all_pp_ranks() and build a dict using global_base_prefix as key for indexing those tasks, then we materialize (finish) those adapters' tasks with base weight conversion tasks in the streaming manner, and store them into a list AdapterWeight. Finally we do LoRA merge using those AdapterWeight.

[yaoyu-33]

/ok to test 37b2a2e

[HollowMan6]

Test results on MoE (Qwen3-30B-A3B) LoRA also looks good:

Canonical LoRA will work on MoE with the following PRs merged:

• [Canonical LoRA] fix: use correct q_out_features for linear_q #1627
• [Canonical LoRA] fix: forward under expert layers #1628

[yaoyu-33]

/ok to test 19c4c5e