Skip to content

[RL] Add host-local checkpoint pull support - #45

Open
aoshen02 wants to merge 839 commits into
mainfrom
codex/pull-checkpoint-api
Open

[RL] Add host-local checkpoint pull support#45
aoshen02 wants to merge 839 commits into
mainfrom
codex/pull-checkpoint-api

Conversation

@aoshen02

@aoshen02 aoshen02 commented Sep 4, 2026

Copy link
Copy Markdown
Owner

Motivation

Vime's disk transport publishes full checkpoints and compressed delta versions to shared storage. Each rollout host needs to materialize the requested version into its own local checkpoint directory before vLLM reloads it.

This is intentionally separate from vllm-project#50723: that PR applies sparse checkpoint-coordinate updates directly to a loaded model, while this change maintains an on-disk host-local checkpoint and exposes it through worker RPC.

Design

  • Add a locked, versioned local checkpoint materializer.
  • Seed from the model checkpoint or the latest published full checkpoint.
  • Apply ordered xor or sparse overwrite zstd deltas in place.
  • Verify per-tensor checksums before advancing local state.
  • Expose Worker.pull_weights for collective_rpc fan-out across rollout hosts.

Tests

  • python -m pytest tests/model_executor/model_loader/test_local_checkpoint.py -q — 4 passed.
  • pre-commit run ruff-check --files ... — passed.
  • pre-commit run ruff-format --files ... — passed.

AI assistance was used to port the existing Vime image patch onto current vLLM main. Every changed line and the focused tests were reviewed by the submitter.

eligotts and others added 30 commits August 30, 2026 23:29
@eligotts @codex
[Frontend] Forward cache salt for content parts (vllm-project#54315)
4ae1722 
Signed-off-by: Eli Gotts <78387377+eligotts@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
@Prudhvivuda
[Bugfix][Multimodal] Honor modality-scoped mm_processor_kwargs (vllm-…
7292ee2 
…project#53808)

Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
@KurodaKanbei @jeejeelee
[Performance] Optimize Dots3 NOTE runtime (vllm-project#53517)
da0b2d8 
Signed-off-by: KurodaKanbei <mistergalahad@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
@jeejeelee
[Kernel] Retire the DSv3 router GEMM CUDA kernel (vllm-project#54040)
8e92248 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
@mkhazraee @NickLucche @DarkLight1337
[KV Offload] Forward ownership in KV cache events (vllm-project#52067)
f9d666f 
Signed-off-by: Moein Khazraee <moein@nvidia.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
@foraxe @drakosha @LucasWilkinson
[Bugfix][DCP] Fix NVIDIA DeepSeek-V3.2 / GLM-5.2 fused attention (vll…
2cf82bc 
…m-project#50005)

Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
@bigPYJ1151
[Bugfix][CPU] Fix several bugs (vllm-project#54042)
fdbf2dd 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
@zufangzhu opencode
[XPU] Ensure unquantized linear weight is N-contiguous (vllm-project#…
2a61f06 
…53536)

Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: opencode <noreply@opencode.ai>
@tianmu-li @codex
[CPU] Support FP16/BF16 persisted GDN state on AMX (vllm-project#52191)
c6c33f2 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
@jl9876 @jikunshang
[Quantization][Autoround][XPU] Support AutoRound MXFP8 MoE models (vl…
1b9539d 
…lm-project#51248)

Signed-off-by: jl9876 <jia.li@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
@noooop
[Renderer] Shutdown the renderer properly. (vllm-project#52124)
5bfd763 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
@AndreasKaratzas
[Bugfix][MM] Fix MiniCPM-o image processor reuse on Transformers v5 (v…
399247c 
…llm-project#54501)

Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
@Hotragn
[Bugfix][Frontend] Truncate prompt_is_token_ids with the prompt (vllm…
28bf75c 
…-project#54509)

Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in>
@lucifer1004 @yewentao256 @mergify
[Bugfix][SM120] DSv4: pass contiguous C128A decode topk indices on SM…
699e180 
…120 (vllm-project#53574)

Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
@liranschour
[Bugfix][KVOffload] P2P tier declares REQUEST_LEVEL on the producer l…
d8de4ae 
…eg (vllm-project#52912)

Signed-off-by: Liran Schour <lirans@il.ibm.com>
@noooop @DarkLight1337
[Frontend] Move engine/protocol.py out openai folder (vllm-project#54492
eeb549a 
)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@mkhazraee
[KV Offload] Preserve KV event metadata until final residency removal (…
9acbc53 
…vllm-project#52068)

Signed-off-by: Moein Khazraee <moein@nvidia.com>
@maithilijoshi20
[Test][Qwen3-VL] Cover compiled DeepStack input contract (vllm-projec…
9debcd5 
…t#53529)

Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
@waizuichougou
[Frontend][Performance] Resolve async media across modalities concurr…
810bc32 
…ently (vllm-project#54537)

Signed-off-by: waizuichougou <2082431897@qq.com>
@Etelis @EtelisIBM @mergify
[KV offload] Order CPU->GPU loads against the compute stream (vllm-pr…
bed3280 
…oject#50696)

Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
@jeejeelee
[kernel] Fused embedding kernel (vllm-project#53677)
e2c8eea 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
@li-ukumar @orozery
[Bugfix][KV Offload][P2P] Preserve aborted loads until abort completi…
e0d2704 
…on (vllm-project#52571)

Signed-off-by: Uttam Kumar <uttam.kumar912@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
@Etelis @EtelisIBM @NickLucche
[Bugfix][KV Offload] Unlink /dev/shm region after all workers map it …
4c58a0c 
…(barrier variant of vllm-project#51317) (vllm-project#52596)

Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
@Etelis @EtelisIBM @orozery
[KV Connector][Offloading] Certify attention-only hybrids in the cano…
c5d840f 
…nical portability gate (vllm-project#51689)

Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
@andyluo7 @codex
[Bugfix][ROCm] Preserve AITER unified-attention metadata during graph…
76ff0cd 
… replay (vllm-project#53821)

Signed-off-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
@NickLucche
[Core] Add max_num_queued_reqs and max_num_queued_tokens for queu…
dafbef1 
…e size management (vllm-project#49445)

Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
@Zhenzhong1 @jikunshang
[AutoRound] Support AutoRound Format Block-Wise FP8 in vLLM (vllm-pro…
bd575a0 
…ject#47434)

Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
Signed-off-by: Zhenzhong Xu <zhenzhong.xu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
@faaany @jikunshang
[Tests][XPU] Limit Qwen2-VL generation length to avoid flaky numerica…
82936c4 
…l divergence (vllm-project#54172)

Signed-off-by: Lin, Fanli <fanli.lin@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
@rasmith @AndreasKaratzas
[ROCm] Add TheRock preview docker updates, Keep Python 3.12 and Ubunt…
3593c96 
…u 22.04 (vllm-project#49925)

Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
@NickLucche
[CI] Exclude kv_transfer changes from broad spec-decode/kernels/multi…
e9dd6d4 
…modal triggers (vllm-project#54365)

Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
majunze2001 and others added 29 commits September 3, 2026 15:16
@majunze2001 @codex
[Bugfix][KV Connector] Safely fill circular buffers in DecodeBench (v…
31e9c13 
…llm-project#54879)

Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: OpenAI Codex <noreply@openai.com>
@njhill
[Core] Deprecate "all" mamba cache mode (vllm-project#55041)
e55b93f 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
@mevince
[Bugfix][KV Connector] Populate SimpleCPUOffload BlockStored metadata (…
4ae6228 
…vllm-project#54325)
@yifjiang
[Bugfix][KV Offload] Do not let a recurrent group's unhashed block tr…
da8ec28 
…uncate the load boundary (vllm-project#52807)

Signed-off-by: Yifan Jiang <19356972+yifjiang@users.noreply.github.com>
@ZJY0516
[Model] add GLM-5.3-Flash support (vllm-project#53906)
98ed085
@BolinSNLHM @claude
[Kernel] Enable Kimi-K3 SiTU on the CuteDSL MoE backend and the SM107…
d410fc1 
… low-latency GEMM plan (vllm-project#54606)

Signed-off-by: Bolin Sun <bolins@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@sychen52 @vadiklyutiy
[Bugfix][Model] Fix FP8 PLE loading in mixed ModelOpt checkpoints (vl…
d4d703c 
…lm-project#54882)

Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
@mgoin @claude @ElizaWszola
[Fusion] Manual ActivationQuantFusionPass initial application (vllm…
b762406 
…-project#51415)

Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
@Fangzhou-Ai
[ROCm][Perf] Add low-M FP32 router GEMM for gfx950 (vllm-project#54845)
8bf3963
@BugenZhao @mergify
[Rust Frontend] Use token-attributed text in reasoning and unified pa…
21a2211 
…rsers (vllm-project#54884)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@wseaton @BugenZhao
[Rust Frontend] Support --lora-modules for static adapter loading (v…
2db1c4d 
…llm-project#54837)

Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Will Eaton <weaton@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@Yuzu23 @huangxh1618 @khluu
Fix DeepSeek V4 FlashMLA auto KV cache dtype (vllm-project#45091)
fc8f107 
Signed-off-by: elehayym <huangxh1618@gmail.com>
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: elehayym <huangxh1618@gmail.com>
Co-authored-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
@andyxning @DarkLight1337
[warmup] overlap renderer warmup and engine core initialization (vllm…
2a336d8 
…-project#54557)

Signed-off-by: Andy Xie <andy.xning@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
@Fangzhou-Ai @AndreasKaratzas @codex
[ROCm][Perf] Optimize MiniMax-M3 decode indexer and top-k (vllm-proje…
cee0f92 
…ct#54682)

Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
@taneem-ibrahim
[CI] Avoid logging test server environment values (vllm-project#54379)
e410111 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
@andy-neuma @Chinmay-Kulkarni-AMD
@cursoragent @claude
[CI] Zen5 image build (vllm-project#50314)
6fdee17 
Signed-off-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com>
Signed-off-by: andy-neuma <alinfoot@redhat.com>
Co-authored-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@zdtsw @BugenZhao Richard
[Rust Frontend] Add support for TLS in render server (vllm-project#54999
c7e6e36 
)

Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Richard <noreply@moonshot.cn>
Signed-off-by: Wen Zhou <wenzhou@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@zyongye @claude
[Perf][Kimi-K3] Cut MLA decode concat/cache epilogue latency (vllm-pr…
9509fc8 
…oject#54896)

Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@zyongye @claude
[Perf] Prefetch the weight before the PDL wait in fused_q_kv_rmsnorm (v…
bc2ee48 
…llm-project#55020)

Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@Rohan138 @claude
@AndreasKaratzas @Fangzhou-Ai
[ROCm] Bump AITER to 0.1.21.post1 (vllm-project#52826)
579aef4 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: fai <fangzhouai@gmail.com>
@BugenZhao
[Rust Frontend] Report reasoning tokens in chat completion usage (vll…
d6bce42 
…m-project#54883)

Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@aoshen02 @claude @vx120
[Perf][Model Runner V2] Compact sampling masks on GPU instead of unpa…
560ef78 
…cking the full-vocab bitmask on CPU (vllm-project#54901)

Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: vx120 <893600387@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: vx120 <893600387@qq.com>
@foraxe @codex
[Bugfix][DCP] Materialize prefill keys on non-owner ranks (vllm-proje…
19c018e 
…ct#54908)

Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
@xuhuan51 liuguangli
@codex @mergify
[Core][KV Events] Echo session_id on GPU BlockStored events (vllm-pro…
d9e2b52 
…ject#51381)

Signed-off-by: liuguangli <liuguangli35@gmail.com>
Signed-off-by: jinx-machinepulse <2679622408@qq.com>
Co-authored-by: liuguangli <liuguangli35@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
@Rohan138 @claude
@AndreasKaratzas @codex
[ROCm][CI] Build and publish TheRock nightly docker images (vllm-proj…
7dc30f5 
…ect#55014)

Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
@bnellnm @tlrmchlsmth
[KVConnector] Add retention interval to OffloadingConnector (vllm-pro…
25268f0 
…ject#51886)

Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
@ZeldaHuang @codex
[Bugfix][PD] Pad resumed speculative decode requests (vllm-project#55126
a26b71d 
)

Signed-off-by: Ziming Huang <48115868+ZeldaHuang@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
@aoshen02
[Performance][DSv4] Size dequant gather launch grid by rows (vllm-pro…
69cf055 
…ject#55061)

Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen524@gmail.com>
@aoshen02 @codex
[RL] Add host-local checkpoint pull support
03fad9f 
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
@aoshen02 aoshen02 mentioned this pull request Sep 9, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.