[1/n] Disaggregated rollout: delta weight sync (disk + NCCL transports)

[nanjiangwill]

Series:

1. [1/n] Disaggregated rollout: delta weight sync (disk + NCCL transports) #1806
2. [2/n] Disaggregated rollout: disk-level delta weight sync #2089
3. [3/n] Disaggregated rollout: engine-side /pull_weights #2181

Summary

Non-colocated weight sync that ships only changed positions + values instead of every parameter. The motivating use case is training/inference disaggregation — trainer and rollout engines in different datacenters over a shared filesystem with bandwidth on the order of 100s of MB/s, where a full broadcast is infeasible but a sparse delta (~3% density, ~5 GB for a 355B model) is. Two transports share one wire layout and one receiver-side decoder:

• disk — write one safetensors file per flush to --update-weight-delta-dir; one HTTP push per sync wakes the rollout engines to read+apply. For cross-DC training/inference disaggregation.
• nccl — broadcast each per-flush bucket directly. Intra-DC validation baseline.

Receiver applies via NaN-masked overwrite — no arithmetic on either side. Lossless by construction; no drift, no periodic base syncs needed.

Inspired by arXiv:2509.19128 (selective overwrite); cross-DC disaggregation motivation from Fireworks AI — Frontier RL Is Cheaper Than You Think.

CLI surface

Trainer side (slime):

flag	values	applies to
--update-weight-mode	full / delta	universal — picks the strategy
--update-weight-transport	nccl / disk	delta only — per-flush carrier
--update-weight-encoding	indices / deltas / deltas_zstd	delta only — position encoding
--update-weight-delta-dir	path	delta + disk — shared-FS root for per-sync directories
--update-weight-delta-keep-files	bool flag	delta + disk — debug aid (skip post-apply cleanup)
--custom-delta-pre-push-path	module.fn	delta + disk — trainer-side hook (e.g., shared-FS commit)

SGLang side (auto-mirrored via --sglang- prefix):

flag	purpose
--sglang-update-weight-delta-chunk-bytes	byte cap per model.load_weights call on apply
--sglang-update-weight-delta-read-workers	max parallel I/O threads for reading delta files (disk only)

--update-weight-mode=delta is rejected with --colocate — CUDA IPC has no wire to compress.

Code shape

Slime: new UpdateWeightFromDistributedDelta extends the refactored UpdateWeightFromDistributed (template-method base with _iter_non_expert_chunks / _iter_expert_chunks / _on_chunk hook; bucketing lives in the iterators). One class for both transports — the if self.transport == "nccl" branch lives only in _flush_bucket and _finalize_sync. Encoders (encode_indices, encode_deltas), snapshot state (DeltaState), and the disk-only writer (AsyncSafetensorsWriter) are module-level.

SGLang patch (to follow in next PR):

• Wire protocol: DeltaSpec + DeltaEncoding + DeltaParam in io_struct.py. Receiver dispatches on load_format == "delta".
• Two receive entry points, both converge on _apply_delta_payload:
  • NCCL: update_weights_from_distributed(load_format="delta", delta=spec) broadcasts (__positions__, __values__) tensors, then applies.
  • Disk: update_weights_from_disk(load_format="delta", model_path=..., files=[...]) reads safetensors files (batched parallel prefetch), then applies.
• Apply: _delta_apply_context patches Tensor.copy_ / fill_ scoped to writes whose destination is inside model param storage. NaN sentinel at unchanged positions triggers masked overwrite. No arithmetic, lossless.

🤖 Generated with Claude Code