Lora Autoscaler #287
Description
What would you like to be added:
Support load/unload loras based on the metrics, just like HPA for pod autoscaling.
part of #27
Why is this needed:
Support N models with M * N user cases.
Completion requirements:
This enhancement requires the following artifacts:
- Design doc
- API change
- Docs update
The artifacts should be linked in subsequent comments.