[https://nvbugs/5510879][fix] Fix pytorch & TRT-python flows fused LoRA adapter modules weight split with TP>1#8063

Merged

amitz-nv merged 16 commits intoNVIDIA:mainfrom

amitz-nv:dev-fix-lora-fused-modules-weights

Oct 12, 2025

Commits on Oct 12, 2025

Interleave mlp_gate_up LoRA weights for TP, move the mapping argument from LoraManager load methods to its constructor, remove other TP references in load_from_model_dir as until now it always rece…
amitz-nv
committed
Refactor fused LoRA module weight interleaving logic into its own function, add support for LorA fused attn QKV, pass ModelConfig to LoraManager ctor for fused QKV LoRA adapter support
amitz-nv
committed
Remove small code duplication
amitz-nv
committed
Minor fix
amitz-nv
committed
Added test_lora_fused_modules_output_on_tp2_identical_to_tp1 to pytorch flow, minor fixes & improvements
amitz-nv
committed
Add inline comment
amitz-nv
committed
Using Python 3.8 type hints
amitz-nv
committed
Remove unnecessary import
amitz-nv
committed
Remove passing of removed runtime_mapping argument in call to lora_manager.load_from_ckpt
amitz-nv
committed
Pass mapping correctly in TRT-python flow as well
amitz-nv
committed
In TRT-python flow use mapping only from engine_config.pretrained_config.mapping
amitz-nv
committed
Refactor 'prepare fused lora modules for TP' logic into its own inner function, pass cpp_peft_cache_manager to LoraManager creation, add 'phi3' to relevant test names
amitz-nv
committed
Convert more fields of ModelConfig class from binding
amitz-nv
committed
Improve comments
amitz-nv
committed
Convert model_config.num_layers from binding class to python class without dividing by PP size
amitz-nv
committed
Mark test to run only when there are 2 GPUs, improve documentation
amitz-nv
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[https://nvbugs/5510879][fix] Fix pytorch & TRT-python flows fused LoRA adapter modules weight split with TP>1#8063

[https://nvbugs/5510879][fix] Fix pytorch & TRT-python flows fused LoRA adapter modules weight split with TP>1#8063
amitz-nv merged 16 commits intoNVIDIA:mainfrom
amitz-nv:dev-fix-lora-fused-modules-weights

Commits on Oct 12, 2025

Interleave mlp_gate_up LoRA weights for TP, move the mapping argument from LoraManager load methods to its constructor, remove other TP references in load_from_model_dir as until now it always rece…

Refactor fused LoRA module weight interleaving logic into its own function, add support for LorA fused attn QKV, pass ModelConfig to LoraManager ctor for fused QKV LoRA adapter support

Remove small code duplication

Minor fix

Added test_lora_fused_modules_output_on_tp2_identical_to_tp1 to pytorch flow, minor fixes & improvements

Add inline comment

Using Python 3.8 type hints

Remove unnecessary import

Remove passing of removed runtime_mapping argument in call to lora_manager.load_from_ckpt

Pass mapping correctly in TRT-python flow as well

In TRT-python flow use mapping only from engine_config.pretrained_config.mapping

Refactor 'prepare fused lora modules for TP' logic into its own inner function, pass cpp_peft_cache_manager to LoraManager creation, add 'phi3' to relevant test names

Convert more fields of ModelConfig class from binding

Improve comments

Convert model_config.num_layers from binding class to python class without dividing by PP size

Mark test to run only when there are 2 GPUs, improve documentation