[Refactor] Move weight completeness verification into the loader

[EazyReal]

Motivation

load_weights(weights) is reused as a tensor-copy primitive by startup loading, online/RL refits, and filtered Elastic-EP recovery. Model-local whole-checkpoint checks therefore fire in contexts that intentionally load only a subset.

The underlying contract is still needed on current main: RFC #28585 remains open, and the Weight Loader v2 tracker #31051 still lists the completeness result and verification contract as unfinished. This PR supplies that boundary without coupling it to one tensor-source implementation.

Changes

• Add opt-in loader-side verification of required model parameters after a full-checkpoint load.
• Let migrated models declare legitimate omissions with is_optional_weight(name).
• Make full-checkpoint context explicit at the loader boundary.
• Skip model-wide verification for filtered Elastic-EP refills and all FLASH_RL refits, including alternate disk transport formats.
• Preserve model-local atomic checks for fused checkpoint shards.
• Migrate Gemma4, DeepSeek-V4 / NextN / OCR, and Llama4 to return loaded parameter names and remove their model-wide diagnostics.
• Cover the default, quantized RL initial-load, BitsAndBytes, GGUF, remote filesystem, and CPU-quantization loader paths.

The verifier intentionally checks parameters, not buffers. Persistent and derived buffer initialization remains model-owned.

Review context

• RFC: [RFC] Weight loading: separate loading from completeness verification #28585
• Weight Loader v2 plan: Weight Loader v2 — protocol and migration plan (PR1–PR6) #31051; initial demo AutoWeightLoader support Sglang native models 1: demo  #28671; expanded migration feat: expand weight loader v2 PR2 coverage #32565
• Loader-only staged slice: [Refactor] Add opt-in loader-side weight completeness verification #28962
• Interim model-local stopgaps: [Fix] Suppress Gemma4 missing-weight logs for partial updates #28519 and [Fix] Scope full-checkpoint diagnostics to complete weight loads #28560

Verification

• Hosted lint passes on head 62d7418510: https://github.com/sgl-project/sglang/actions/runs/30517415886
• All repository pre-commit hooks pass on the touched files.
• git diff --check passes.
• python3 -m py_compile passes on every touched Python file.
• The isolated completeness helper contract passes.
• A fresh-context independent review passes with no unresolved findings.
• test/registered/unit/model_loader/test_load_and_verify_weights.py is registered for CPU CI. The full unit test could not run in this checkout because its Python environment does not include the project dependencies; hosted test matrices require the maintainer-controlled run-ci label.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[EazyReal]

@kpham-sgl @pyc96 @b8zhong Could I get a quick review and run-ci on this loader/verifier split? It keeps load_weights as the tensor-copy primitive and moves completeness checks to full-checkpoint loading.

[EazyReal]

@b8zhong could you take a look at this loader-side weight completeness refactor if you have bandwidth? It moves full-checkpoint checks out of load_weights, which should keep partial updates safe; a run-ci label would also help if it looks reasonable.

[EazyReal]

@b8zhong @JD-ETH just rebased this. The Weight Loader v2 plan still has “Completeness result and verification contract” open; this PR makes that full-checkpoint decision loader-owned while preserving load_weights for partial FLASH_RL and Elastic-EP updates.