[II] Own borrowed weights before deferred online processing

[voipmonitor]

Status

• Implementation: implemented
• Unit qualification: qualified
• GLM-5.2 EXL3 E2E qualification: qualified for the profile below

Purpose

Preserve weight values when layerwise online quantization defers a parameter
loader while InstantTensor uses borrowed staging-buffer views.

Resulting behavior

make_online_process_loader() may retain bound loader arguments until every
parameter required by the layer is available. A tensor marked
_vllm_instanttensor_borrowed is cloned before entering that deferred state.
Unmarked tensors and loaders that consume their inputs synchronously retain
their allocation behavior.

The owned copy is scoped to the incomplete layer and released after online
processing. This avoids the checkpoint-wide memory cost of
INSTANTTENSOR_COPY=1 while preventing staging-buffer reuse from changing a
deferred weight.

Technical reason

InstantTensor borrowed mode permits the loader to reuse backing storage after
the iterator advances. Layerwise online processing retained the borrowed view
in LayerReloadingInfo.loaded_weights. A later tensor load could overwrite a
parameter before the layer's online quantizer consumed it.

Pull request #281 defines the borrowed-buffer contract and requires retaining
consumers to materialize owned storage. This change enforces that contract at
the layerwise deferred-loader ownership boundary.

Compatibility

The clone is gated by the explicit borrowed-tensor marker. Safetensors,
owned-copy InstantTensor loads, ordinary parameter loaders, and completed
layers do not acquire an additional copy.

Validation

Focused tests in the Infernal Invocation CUDA 13.3 / PyTorch 2.13 image:

python -m pytest -q --confcutdir=/tmp /tmp/test_reload.py \
  -k 'online_processing or (layerwise_reload and not marlin)'
8 passed, 38 deselected

Ruff check, Ruff format, and git diff --check passed for the implementation
and tests.

Exact-image E2E qualification used:

image: voipmonitor/vllm:infernal-invocation-vllm908522a-b12x5d648d9-fi1ac6942-cu133-torch213-20260813-r11
digest: sha256:01b973d1ae132882bcc1bf62ea232f6aabe649dd4a89b961d81f3c41cc53f971
checkpoint: brandonmusic/GLM-5.2-EXL3-TR3v4-3.5bpw-MTP78
revision: 9ab9579774cc432df91567a36f6e9e863e0d4c9f
configuration: TP4/DCP1/MTP3, online K6, NVFP4 DS-MLA KV,
               InstantTensor BUFFERED COPY=0, FULL decode graphs,
               MNS8, graph cap 32, max model length 65,536

• The 322 GiB checkpoint loaded and every mixed K3/K4/K5 routed-expert layer
  completed preparation.
• A deterministic arithmetic request returned 703 for 37 * 19.
• Two warmed CC1/context-zero, 20-second measurements reached 131.45 and
  127.22 aggregate tok/s with zero request errors.
• No source file was mounted over the published image.

Machine-readable evidence:
https://github.com/local-inference-lab/blackwell-llm-docker/blob/6f92a9ecff35f0dabd6f444b05daab4024b49257/validation/infernal-invocation-r11-local-gpu.json

Duplicate-work check

Open local and upstream pull requests and issues were searched for
InstantTensor borrowed-buffer ownership in deferred layerwise processing. No
overlapping implementation was found. Pull request #281 is the prerequisite
producer contract rather than a duplicate consumer-side fix.

Qualification boundary

The E2E result qualifies the checkpoint revision and TP4/DCP1/MTP3 profile
above. Other retained consumers still require their own ownership audit; this
pull request changes only layerwise online processing.

AI assistance

AI assistance was used for investigation, implementation, validation, and
pull-request text. The human submitter must review and be able to defend every
changed line before merge.

[coderabbitai]

Warning

Review limit reached

@voipmonitor, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 49 minutes

Limit details: You’ve used all 1 included review currently available under your plan.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1dbd390e-6438-4d4f-b585-3202247286ad

📥 Commits

Reviewing files that changed from the base of the PR and between ce5f50f and a5389de.

📒 Files selected for processing (2)

• tests/model_executor/model_loader/test_reload.py
• vllm/model_executor/model_loader/reload/layerwise.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

Superseded by the deferred accelerator-tensor ownership rule merged through #382. Commit 6a5c183c7e88e8e250e80ca44c7fdf21fae5cbb3 clones every non-CPU, non-meta tensor retained by the layerwise deferred loader, including InstantTensor borrowed views and mapped shard views. That implementation preserves the ownership invariant covered by this pull request with broader tests on the canonical dev/infernal-invocation branch.