Log KV cache GiB usage and warn when max_num_seqs exceeds capacity#38408
Open
ashishkamra wants to merge 1 commit intovllm-project:mainfrom
Open
Log KV cache GiB usage and warn when max_num_seqs exceeds capacity#38408ashishkamra wants to merge 1 commit intovllm-project:mainfrom
ashishkamra wants to merge 1 commit intovllm-project:mainfrom