Skip to content

[KV offload] Add max_recompute_pending_tokens to skip waiting on short in-flight offloads - #57959

Open
robertgshaw2-redhat wants to merge 2 commits into
vllm-project:mainfrom
robertgshaw2-redhat:claude/fervent-wright-129orl
Open

[KV offload] Add max_recompute_pending_tokens to skip waiting on short in-flight offloads#57959
robertgshaw2-redhat wants to merge 2 commits into
vllm-project:mainfrom
robertgshaw2-redhat:claude/fervent-wright-129orl

Commits

  1. Commits on Sep 21, 2026