feat: ProTrain integration with BlockMode.OFFLOAD (Option B complete)#18

Closed

thad0ctor wants to merge 143 commits into

mainfrom

protrain-optim-checkpoint-phase2-mode-c

Commits on Apr 23, 2026

Commits on Apr 24, 2026

tests: harden 7B capacity-safety, add SWAP/monotonicity/multi-GPU-derate coverage, implement zombie skips

thad0ctor
and
claude
committed
chunk: fix CPU-Adam race, view-dtype alignment, adapter order, data repointing; α=1.10

thad0ctor
and
claude
committed
plugin: wire create_optimizer dispatch, broaden mutex, fix defaults + docstring + YAML

thad0ctor
and
claude
committed
profiler: record per-op latencies; cost model uses measured compute; tighten 7B runtime tolerance

thad0ctor
and
claude
committed
M7: true ZeRO-3 chunk sharding (all_gather / reduce_scatter)

thad0ctor
and
claude
committed
M7 followup: cost-model sharding awareness + mixed-dtype shard support

thad0ctor
and
claude
committed
bench: multi-GPU throughput comparison (DDP / replicated / ZeRO-3)

thad0ctor
and
claude
committed
plugin: auto-select multi-GPU mode (A/B/C) based on workload fit + CPU RAM

thad0ctor
and
claude
committed
profiler: add CPU+GPU Adam microbenchmarks; loosen 7B runtime tolerance

thad0ctor
and
claude
committed
profiler: measure hook-less steady-state wall time; cost model scales by ratio

thad0ctor
and
claude
committed
M7 cost-model close-out: PCIe plumb-through + steady-state cap + asymmetric peak tolerance

thad0ctor
and
claude
committed
profiler: record steady_fwd_peak_bytes; memory cost model caps at measured peak when configs are all-NONE

thad0ctor
and
claude
committed
profiler: record per-block steady peaks; memory cost model uses them as ground-truth caps

thad0ctor
and
claude
committed
cost+search: extract hot_iter_peak_cap helper; plumb into searcher's fast path

thad0ctor
and
claude
committed
profiler: multi-iter hot-loop steady measurement; cost model uses measured bwd/fwd ratio

thad0ctor
and
claude
committed

Commits on Apr 26, 2026

Commits on Apr 27, 2026

Commits on Apr 28, 2026

Commits on Apr 29, 2026

test(protrain): fix flaky tiny_llama loss check + 4gpu MASTER_PORT collision

thad0ctor
and
claude
committed

Commits on May 1, 2026

Commits on May 4, 2026

Commits on May 5, 2026