[Kernel] Full Tensor Parallelism for LoRA Layers#3524

Merged

Yard1 merged 18 commits intovllm-project:mainfrom

FurtherAI:lora_full_tensor_parallel

Apr 27, 2024

Commits on Mar 20, 2024

Commits on Mar 29, 2024

Reduce duplicate code in the new layers.
Austin
committed

Commits on Mar 30, 2024

Add the ability to select fully sharded loras for the user. Propose changes to test_layers which should test the new layers with pytest.
Austin
committed
Modify for formatting compliance.
Austin
committed

Commits on Apr 16, 2024

Added docstrings, cleaned up code, mainly set_lora() for the layers.
Austin
committed

Commits on Apr 20, 2024

Added tests for extra punica shapes.
Austin
committed
[Merge] Merged main
Austin
committed
[Misc] Fixed a typo
Austin
committed

Commits on Apr 26, 2024

Commits on Apr 27, 2024

[Misc] Formatting.
Austin
committed
[Misc] Formatting, local formatter and PR test cannot agree.
Austin
committed
Disable yapf for block
Yard1
authored
Lint
Yard1
authored