Explicit pool claim/reset: one trainer call owns one pool epoch

[jvmncs]

Summary

Makes pool ownership explicit and restart-safe: one trainer launch ↔ one run ↔ one pool epoch. Today the pool's reset-to-base is inferred from a run_id mismatch on the first publish, and two cookbooks write latest two different ways — so a crash-restart that reused a run_id (derived from a dir basename) could leave the pointer pinned to a dead incarnation's high-water version, and "who owns pool state" was genuinely ambiguous.

This PR introduces a single pointer-control vocabulary all writers share, an explicit claim (write the empty pointer at launch), and a minimal no-Modal harness that pins the invariants down with tests.

Shared decision in protocol.py

One rule both writer paths go through, so they can't bake in divergent semantics:

BASE_VERSION = 0  # the "empty" pointer a launch claims (forks at base)

def decide_pointer_move(current_run_id, current_version, *, run_id, version) -> PointerMove:
    if run_id != current_run_id:
        return PointerMove(run_id, version, reset=True)   # new run forks at base (incl. claim)
    if version <= current_version:
        raise PointerRewind(...)                          # monotonic within a run
    return PointerMove(run_id, version, reset=False)

Board: guarded writers (bulletin.py)

board.advance(run_id, version)  # decide_pointer_move + write_latest; raises PointerRewind on rewind
board.claim(run_id)             # advance(run_id, BASE_VERSION): write <run_id>/weight_v000000

claim resets every replica to base before any delta. The reconcile path is unchanged — the pool still resets on a run_id switch via WeightSyncManager._switch_run (re-materialize base under the commit gate).

Consolidation (de-divergence)

• frontdoor advance_latest_decision now delegates to decide_pointer_move (only parses the wire identity + shapes the error dict). modal_serve.advance_to writes through board.write_latest instead of a third raw latest write.
• bulletin_hooks commit_and_wake publishes via board.advance (monotonic; new-run first publish forks at base) instead of an unconditional write_latest; adds claim_pool (rank-0 launch hook: claim → commit volume → wake). slime/miles hooks.py get thin claim_pool wrappers and their modal_train launches call it before the trainer starts publishing.
• _run_id is now required (passed via custom_config) — the dir-basename fallback (the reuse hazard) is gone. Both disagg trainers already mint a fresh uuid4 per launch.

A restart is now just a new epoch: a fresh run_id claims and resets the pool; reusing a run_id raises PointerRewind instead of silently serving stale weights.

Minimal config: cookbook/local_disagg/

No Modal, no slime/miles, no GPUs — a filesystem board, an in-memory pool (MemoryEngine + real WeightSyncManager), and a trainer-as-writer using the same claim/advance/reconcile primitives. harness_test.py asserts the invariants:

• claim resets the pool to base (v0) under the new run's id
• within a run the pool converges to each published version in order
• a new run forks at base even from a higher prior version (engine reset observed)
• re-claiming a reused run_id is rejected as a rewind; a fresh run_id claims clean
• non-monotonic publish within a run is rejected
• a late-joining (cold/scaled-up) replica reconciles to the current run

Testing

uv run python -m pytest -q → 98 passed (was 82; +16 new tests across protocol_test, bulletin_test, frontdoor_test, and the new harness_test). Modal-only cookbook files (modal_train.py, hooks, modal_serve.py) py_compile-check clean. No lint config or CI workflows exist in the repo; pytest is the gate.

Link to Devin session: https://modal.devinenterprise.com/sessions/b0d6110c71924c24943bcb5cea1eb5f4
Requested by: @jvmncs

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR that start with 'DevinAI' or '@devin'.
• Look at CI failures and help fix them

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.