[Bugfix] Size the extensible KV cache from UMA-corrected free memory - #65413

[Bugfix] Size the extensible KV cache from UMA-corrected free memory#65413

njhill merged 6 commits into

vllm-project:mainvllm-project/vllm:mainfrom

aijanai:amassidda/extensible-kv-uma-free-memoryaijanai/vllm:amassidda/extensible-kv-uma-free-memoryCopy head branch name to clipboard