Skip to content

prime-runs - #2415

Merged
kcoopermiller merged 33 commits into
mainfrom
cooper/prime-runs
Sep 8, 2026
Merged

kcoopermiller merged 33 commits into
mainfrom
cooper/prime-runs

eval: close out under a shield; an in-flight close-out owns the outcome

9706c82
Select commit
Loading
Failed to load commit list.
MacroscopeApp / Macroscope - Correctness Check succeeded Sep 4, 2026 in 2m 52s

No issues identified (3 code objects reviewed).

• Reviewed files modified since 10fea3a; other PR files not modified since then were skipped.
• Merge Base: ffa3291
• Head: 9706c82
• Detection Mode: Balanced (Learn more)

Details

✅ File Path U3 Bytes Comments Posted Reason
✅ verifiers/v1/cli/eval/runner.py 1936 0
✅ verifiers/v1/utils/platform.py 713 0

Billed Total: 10.00KB of diff | $0.50 (This review was charged at our per-review byte minimum of 10.00KB. Learn more here)

Filtered Issues Details

verifiers/v1/cli/eval/runner.py
  • line 243: on_complete awaits asyncio.to_thread(log_episodes, ...) at line 243, but gather_rollouts cancels its rollout task when a sibling fails. Cancelling that await marks the rollout task complete without stopping an already-running executor call, so the cleanup at lines 281-282 can abort/close run while that log_episodes call is still handing off an episode. This reintroduces uploads racing the terminal close and can leave successfully completed, on-disk episodes absent from the streamed platform run. Keep/await the upload futures during cancellation before closing the run. [ Already posted ]