[KV Offload] Add per-request max_load_tokens control

[albertoperdomo2]

Purpose

This PR adds the experimental per-request kv_transfer_params["max_load_tokens"], analogous to max_offload_tokens.

• 0 disables external KV loading, so missing tokens are recomputed.
• A positive value caps how many tokens can be loaded beyond the GPU-resident prefix. The cap is rounded down to a supported KV-cache boundary.
• Omitting the field preserves uncapped loading. Invalid values are ignored with a warning.

Local GPU prefix-cache reuse and KV stores remain enabled.

kv_load_tiers keeps its existing semantics: it filters secondary tiers, while CPU remains available as a direct source and as the staging tier for secondary loads. skip_reading_prefix_cache is unchanged.

AI assistance (Codex) was used to inspect the implementation and draft tests and documentation.

Test Plan

• Test max_load_tokens: 0 with synchronous and asynchronous scheduling.
• Test positive caps, partial tails, alignment, and invalid values.
• Verify that the store path and skip_reading_prefix_cache remain unchanged.
• Run the focused tests on the final revision.
• Validate the final API in a live multi-tier deployment.

Test Result

Pending.

---

Essential Elements of an Effective PR Description Checklist

• Purpose
• Test plan
• Test results
• Documentation update

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

Documentation preview: https://vllm--55885.org.readthedocs.build/en/55885/

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @albertoperdomo2.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[orozery]

Thanks @albertoperdomo2 !
I think that if we want to allow disabling loading from any kind of offloading it's maybe better to introduce a new max_load_tokens (similar to max_offload_tokens).
kv_load_tiers assumes CPU is always included since all other tiers promote through it, so it's meaningless to have a non-empty list without CPU included.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @albertoperdomo2.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[albertoperdomo2]

@orozery can we kick off the CI?

[orozery]

/ci run

[github-actions]

❌ This PR is 1 commit behind upstream main. Your branch must contain every commit currently on upstream main. No new CI build was started. Merge or rebase onto the latest main, then rerun /ci run. To test this branch at your own risk, use /ci run --allow-stale.

[github-actions]

✅ @albertoperdomo2, CI is now available for this PR.

• /ci run starts upstream CI; /amd-ci run starts AMD CI only.
• Your branch must contain every commit currently on its upstream target branch. Merge or rebase onto the latest target branch, then rerun the command. Append --allow-stale to a run command to test an outdated branch at your own risk.
• /ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
• /amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
• /ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.

[albertoperdomo2]

/ci run

[github-actions]

✅ Triggered Buildkite CI #89080 for commit 7b5253a9bf2d.

[albertoperdomo2]

/ci retry

[github-actions]

✅ No failed, timed-out, or expired jobs need retrying: https://buildkite.com/vllm/ci/builds/89080

[albertoperdomo2]

/ci run

[github-actions]

✅ Triggered Buildkite CI #89192 for commit a5d9908b6f90.