[Sync][E] delta weight sync (slime #1806/#1946/#1991)

[aoshen02]

PR-E — delta weight sync (RFC #107). Stacked on PR-A (#145) (base sync/slime-mega-A; touches actor.py which A also touches).

Bandwidth-optimized RL weight sync: trainer bytewise-diffs weights vs a pinned-CPU snapshot, ships only changed positions+values (nccl broadcast / disk safetensors); receiver overwrites only the changed bytes (lossless, NaN-masked). For large / cross-DC non-colocate (colocate uses faster CUDA-IPC, doesn't need delta).

Included

• delta_io.py — DeltaEncoding/DeltaParam/DeltaSpec (local; slime imported these from sglang's io_struct).
• update_weight_from_distributed_delta.py — trainer encoder (subclasses UpdateWeightFromDistributed).
• delta_receiver.py — engine-agnostic pure-torch decode + NaN-masked apply: delta_apply_context monkey-patches torch.Tensor.copy_/fill_ scoped to model param storage, wrapping a normal model.load_weights so only changed (non-NaN) positions are written; post_load_weights (fp8 scales) uses original copy_.
• vLLMColocateWorkerExtension — collective_rpc-callable delta receivers (nccl: recv positions/values on the model_update_group; disk: read safetensors).
• args (--update-weight-mode/transport/encoding/delta-dir/keep-files/chunk-bytes), actor delta-dispatch (lazy import, #1946 backward-compat) + zero-delta version bump.
• 中文 docs docs/zh/advanced/delta-weight-sync.md (+toctree) + e2e test test_delta_weight_update.py (registered in e2e-test-short).

🎉 No image rebuild

slime put the receiver in a build-time sglang.patch. vime injects it at runtime via the worker-extension / the existing _VLLMHijack mechanism (loaded by pip install -e .) — no Dockerfile change, no gb200/h200 rebuild.

Skipped / caveat

• #1993 (sglang EAGLE draft-worker delta forwarding): N/A — no vime/vLLM analogue.
• Caveat (documented): vLLM does NOT refresh EAGLE/MTP draft weights on ANY weight sync (gpu_model_runner.reload_weights loads main-model named_parameters only; drafter loaded once at init). This is a vLLM-wide limitation, not delta-specific (full sync has it too) — to track upstream, not fixed here.

CI

test_delta_weight_update.py (4-GPU non-colocate, disk transport, asserts delta safetensors written) registered in e2e-test-short (.j2+.yml). py_compile clean.

🤖 Generated with Claude Code

---

Fidelity ledger (audited 2026-06-04)

op	detail
Ported	#1806/#1946/#1991 — delta weight encode/decode (engine-agnostic core)
Translated (sglang→vllm)	worker-extension-cls injected into the vLLM worker (apply_delta_from_disk / apply_delta_from_distributed)
Dropped/Deferred	none
Verified	test_delta_weight_update EXIT_RC=0 on rebuilt FCBAGDE (gb200 8910)

[gemini-code-assist]

Code Review

This pull request introduces Delta Weight Sync to optimize bandwidth by transmitting only the byte-level changes of model weights since the last synchronization, supporting both NCCL and disk-based transports. The review feedback highlights three critical robustness issues: a potential RuntimeError when calling .view() on non-contiguous tensors in _bytewise_diff_mask, and two potential crashes when handling non-floating-point (integer) tensors, specifically when initializing a tensor with NaN in decode_delta_one_param and when calling torch.isnan inside the patched copy and fill operations.

[gemini-code-assist]

The _bytewise_diff_mask function calls .view() on current and snapshot without ensuring they are contiguous. If either tensor is non-contiguous (which can happen for quantized weights or sliced buffers), PyTorch will raise a RuntimeError. Adding .contiguous() before .view() ensures safety.

Suggested change

	def _bytewise_diff_mask(current: torch.Tensor, snapshot: torch.Tensor) -> torch.Tensor:
	"""
	Per-element bool mask: True where current and snapshot bytes differ. Dtype-agnostic via view-as-integer.
	"""
	es = current.element_size()
	int_dtype = {1: torch.uint8, 2: torch.int16, 4: torch.int32, 8: torch.int64}.get(es)
	if int_dtype is None:
	raise ValueError(f"unsupported element size {es}")
	return current.view(int_dtype) != snapshot.view(int_dtype)
	def _bytewise_diff_mask(current: torch.Tensor, snapshot: torch.Tensor) -> torch.Tensor:
	"""
	Per-element bool mask: True where current and snapshot bytes differ. Dtype-agnostic via view-as-integer.
	"""
	es = current.element_size()
	int_dtype = {1: torch.uint8, 2: torch.int16, 4: torch.int32, 8: torch.int64}.get(es)
	if int_dtype is None:
	raise ValueError(f"unsupported element size {es}")
	return current.contiguous().view(int_dtype) != snapshot.contiguous().view(int_dtype)

[gemini-code-assist]

decode_delta_one_param attempts to initialize flat with float("nan") for any parameter dtype. If the parameter is an integer tensor (e.g., in quantized models or integer buffers), PyTorch will raise a RuntimeError because NaN cannot be represented in integer types. We should explicitly check and raise a clear error for non-floating-point parameters.

Suggested change

	param_dtype = p.dtype if isinstance(p.dtype, torch.dtype) else getattr(torch, p.dtype)
	flat = torch.full((numel,), float("nan"), dtype=param_dtype, device=device)
	param_dtype = p.dtype if isinstance(p.dtype, torch.dtype) else getattr(torch, p.dtype)
	if not param_dtype.is_floating_point:
	raise ValueError(
	f"Delta weight sync is only supported for floating-point parameters, "
	f"but parameter {p.name} has non-floating-point dtype {p.dtype}."
	)
	flat = torch.full((numel,), float("nan"), dtype=param_dtype, device=device)

[gemini-code-assist]

patched_copy_ and patched_fill_ intercept copies and fills to model parameters/buffers during the delta apply context. However, if any of these parameters/buffers are non-floating-point (such as integer buffers), calling torch.isnan on them will raise a RuntimeError. Restricting the NaN-masking logic to floating-point tensors only prevents these crashes.

    def patched_copy_(self, src, *args, **kwargs):
        if is_param_target(self) is not None and torch.is_floating_point(self):
            src_aligned = (
                src.to(device=self.device, dtype=self.dtype) if src.dtype != self.dtype else src
            )
            mask = ~torch.isnan(src_aligned)
            self[mask] = src_aligned[mask]
            return self
        return original_copy_(self, src, *args, **kwargs)

    def patched_fill_(self, value):
        if is_param_target(self) is not None and torch.is_floating_point(self):
            # NaN scalar means "don't change the param" (per-element analog of
            # patched_copy_). Non-NaN scalars write through.
            try:
                if math.isnan(value):
                    return self
            except TypeError:
                pass
            return original_fill_(self, value)
        return original_fill_(self, value)