[TRTLLM-5830][feat] Improve LoRA cache memory control#6220

Merged

shaharmor98 merged 21 commits intoNVIDIA:mainfrom

amitz-nv:dev-improve-pytorch-lora-cache-memory-control

Jul 31, 2025

Commits on Jul 30, 2025

Remove deprecated lora args from BaseLlmArgs, using peft_cache_config and allowing lora_config.max_loras and lora_config.max_cpu_loras to override it, changed their default value to None
amitz-nv
committed
Enabled use of LoraConfig in TRT_python flow, added tests of expected use of LoraConfig and PeftCacheConfig in LLM args
amitz-nv
committed
Improve comments in tests
amitz-nv
committed
Correct mistake in PeftCacheConfig.num_device_module_layer description
amitz-nv
committed
Add validation of unsupported field in peft cache manager
amitz-nv
committed
Fix docstring line length
amitz-nv
committed
Fix validate_peft_cache_config
amitz-nv
committed
Fix validate_peft_cache_config formatting
amitz-nv
committed
Fix lora_prefetch_dir description and 'unsupported warning' message, clarify test
amitz-nv
committed
Fix tests to configure lora cache size by number of adapters for test stability
amitz-nv
committed
Fix tests to API update - use LoraConfig instead of base LLM args for LoRA args
amitz-nv
committed
Fix tests to explicitly configure lora_config's max_loras and max_cpu_loras for stability
amitz-nv
committed
Define default values in PeftCacheConfig model class for device_cache_percent and host_cache_size, improve device_cache_percent description
amitz-nv
committed
Add default value to description
amitz-nv
committed
Fix PeftCacheConfig.create_from_pybind after changing python fields to be non-optional with default values
amitz-nv
committed
Fix examples/llm-api/llm_multilora.py - use one LoraConfig
amitz-nv
committed
Fix examples/llm-api/llm_multilora.py to not use BuildConfig that's irrelevant to pytorch backend
amitz-nv
committed
Changed create_from_pybind method to be a more generic classmethod in PybindMirror, updated its PeftCacheConfig tests accordingly, removed default values from description, raise exception when unus…
amitz-nv
committed
Minor docstring fix
amitz-nv
committed
Fix rename
amitz-nv
committed
Fix test_ptp_quickstart_multimodal_phi4mm - for stability set lora cache sizes, fix incorrect lora request creation
amitz-nv
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[TRTLLM-5830][feat] Improve LoRA cache memory control#6220

[TRTLLM-5830][feat] Improve LoRA cache memory control#6220
shaharmor98 merged 21 commits intoNVIDIA:mainfrom
amitz-nv:dev-improve-pytorch-lora-cache-memory-control

Commits on Jul 30, 2025

Remove deprecated lora args from BaseLlmArgs, using peft_cache_config and allowing lora_config.max_loras and lora_config.max_cpu_loras to override it, changed their default value to None

Enabled use of LoraConfig in TRT_python flow, added tests of expected use of LoraConfig and PeftCacheConfig in LLM args

Improve comments in tests

Correct mistake in PeftCacheConfig.num_device_module_layer description

Add validation of unsupported field in peft cache manager

Fix docstring line length

Fix validate_peft_cache_config

Fix validate_peft_cache_config formatting

Fix lora_prefetch_dir description and 'unsupported warning' message, clarify test

Fix tests to configure lora cache size by number of adapters for test stability

Fix tests to API update - use LoraConfig instead of base LLM args for LoRA args

Fix tests to explicitly configure lora_config's max_loras and max_cpu_loras for stability

Define default values in PeftCacheConfig model class for device_cache_percent and host_cache_size, improve device_cache_percent description

Add default value to description

Fix PeftCacheConfig.create_from_pybind after changing python fields to be non-optional with default values

Fix examples/llm-api/llm_multilora.py - use one LoraConfig

Fix examples/llm-api/llm_multilora.py to not use BuildConfig that's irrelevant to pytorch backend

Changed create_from_pybind method to be a more generic classmethod in PybindMirror, updated its PeftCacheConfig tests accordingly, removed default values from description, raise exception when unus…

Minor docstring fix

Fix rename

Fix test_ptp_quickstart_multimodal_phi4mm - for stability set lora cache sizes, fix incorrect lora request creation

[TRTLLM-5830][feat] Improve LoRA cache memory control#6220shaharmor98 merged 21 commits intoNVIDIA:mainNVIDIA/TensorRT-LLM:mainfrom amitz-nv:dev-improve-pytorch-lora-cache-memory-controlamitz-nv/TensorRT-LLM:dev-improve-pytorch-lora-cache-memory-controlCopy head branch name to clipboard

Commits

Commits on Jul 30, 2025

[TRTLLM-5830][feat] Improve LoRA cache memory control#6220
shaharmor98 merged 21 commits intoNVIDIA:mainfrom
amitz-nv:dev-improve-pytorch-lora-cache-memory-control