[II] Own deferred accelerator weight tensors

[voipmonitor]

Behavior

Layerwise online processing gives every deferred accelerator tensor independent
storage before queuing its normalized weight-loader arguments. The destination
parameter is never cloned, and CPU and meta tensors retain their existing
storage.

Technical reason

Streaming checkpoint loaders can expose accelerator views backed by a reusable
staging ring. Layerwise processing retains weight-loader arguments until all
checkpoint shards required by a layer arrive. Retaining a borrowed view across
another iterator step allows the staging ring to overwrite data that has not
yet reached the quantizer.

The ownership boundary is applied only where weight-loader execution is
deferred. Synchronous parameter loading remains zero-copy.

Compatibility

• Applies to every streaming loader that supplies accelerator tensors to
  layerwise online processing; it is not model-specific.
• Preserves the existing loader signature and return behavior.
• Does not change CPU checkpoint loading or synchronous full-checkpoint loading.
• Does not change InstantTensor buffer sizing or its default copy policy.

Validation

• Ruff formatting and lint checks pass for both changed files.
• Python bytecode compilation passes.
• A CUDA test verifies that queued source storage differs from the borrowed
  source while tensor values remain exact.
• A CUDA integration test streams five 4 MiB BF16 tensors through an 8 MiB
  InstantTensor ring, forcing multiple wraps; every retained deferred tensor
  remains exact after iteration completes.
• A CPU test verifies that the destination parameter and CPU source tensor keep
  their original identities.

Test result: 3 passed on one NVIDIA GPU in the source-locked Kimi-K3 runtime
image; 5 unrelated tests were deselected.

[coderabbitai]

Warning

Review limit reached

@voipmonitor, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 40 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 62161dbb-40ef-4f32-8669-a525fe7a10e1

📥 Commits

Reviewing files that changed from the base of the PR and between ad848fc and 2ef2ee2.

📒 Files selected for processing (2)

• tests/model_executor/model_loader/instanttensor_loader/test_weight_utils.py
• vllm/model_executor/model_loader/reload/layerwise.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

The behavior implemented by this pull request is preserved in vLLM #382 as commit 1d2bf15d12d4. The stable patch ID differs because the InstantTensor ownership test is integrated after the bounded-staging test. The consolidated range-diff retains the pull request contract. Review and merge #382; this pull request is closed to avoid duplicate review.