[TRTLLM-14692][feat] Overlap LoRA and base model computations - #16951

Merged

Funatiq merged 14 commits into

NVIDIA:mainfrom

AlessioNetti:lora-cuda-stream-overlap

Aug 24, 2026

Commits on Aug 21, 2026

Overlap LoRA computations with base model projections
AlessioNetti
committed
9fc0458
Refactor changes for concurrent LoRA execution
AlessioNetti
committed
f24c415
Encapsulate LoRA output handling inside forward_with_base(), switch to empty_like() for init
AlessioNetti
committed
cea9c3c
Always use in-place add_() to combine base and LoRA outputs
AlessioNetti
committed
88a1fb3
Add overlap_lora_and_base flag to toggle aux CUDA stream
AlessioNetti
committed
a6deba8
CodeRabbit nitpicks and linting fixes
AlessioNetti
committed
0166480
Disable LoRA overlap by default, add focused tests
AlessioNetti
committed
6101420
Move ownership of aux LoRA CUDA stream to CudaGraphLoraManager
AlessioNetti
committed
6380bb9
Call record_stream before lora_output add
AlessioNetti
committed
0405054
Change default for overlap_lora_and_base in CudaGraphLoraManager to False
AlessioNetti
committed
32d3f7f
Add focused test for LoraLayer.forward_with_base()
AlessioNetti
committed
1d0572f
Use add_lora_result() in forward_with_base()
AlessioNetti
committed
b239a4e
Use in-place add_() in add_lora_result()
AlessioNetti
committed
b1822cb
More robust check for lora output type
AlessioNetti
committed
2a99451