[mikasenghaas]

Summary

vf-eval no longer pushes runs to the Prime Intellect platform, and verifiers no longer depends on prime-runs. Platform uploads belong to prime-rl's eval. Hosted evaluations already use it: production and dev set HOSTED_EVAL_V1_RUNNER: prime-rl.

• Config. EvalConfig.push (--push / --no-push) and RunConfig.attach (--run.attach) are removed, and so is the validator that tied them together.
• Run id. RunConfig.id is now a local uuid4().hex, set when the config is created. Before, open_run assigned it from the prime-runs handle. It still goes on every trace (EvalRunInfo) and into $VF_RUN_ID.
• Runner. run_eval no longer opens, streams to, finishes or aborts a platform run. Episodes still go to traces.jsonl as each one finishes.
• Dashboard. The "Pushing traces / Traces pushed" footer line is removed.
• Removed: verifiers/v1/utils/platform.py (PushState, open_run, log_episodes, finish_run, abort_run, and the unused build_samples re-export) and its tests.
• Dependencies. prime-runs is removed from the dependencies and from exclude-newer-package. prime-traces is removed from exclude-newer-package, because nothing pulls it in any more. uv.lock drops both packages and changes nothing else.
• Docs and tests. --no-push is removed from the docs, the nemo-gym-weather README and the examples integration test.

Breaking

• --push, --no-push and push in eval configs are removed. vf-eval runs no longer show up on the platform's Evaluations tab. To upload evals, run prime-rl's uv run eval. Remove --no-push from scripts.
• --run.attach / run.attach is removed. The platform's verifiers v1 runner (generate_v1_eval_script, used when HOSTED_EVAL_V1_RUNNER=verifiers) passes it. Production and dev both use the prime-rl runner, so nothing live calls it. Do not switch a deployment back to the verifiers runner after this ships. The platform's config.py default is still "verifiers", and public-docs verifiers/v1/evaluation.mdx still documents run.attach.
• Saved run configs with a push key fail to load. Every run saved before this change has "push": true in configs/resolved/eval.json, so vf-eval @ <run>/configs/resolved/eval.json --resume fails with "Extra inputs are not permitted". Delete the push key from that file, then resume.

Verification

• pytest tests/unit: 244 passed, 10 skipped (9 need PRIME_API_KEY, 1 needs harbor).
• ruff check, ruff format and ty pass.
• vf-eval reverse-text --env.agent.harness.id null --env.agent.runtime.type subprocess -n 2 -r 1 --no-rich: 2 episodes were written, and both traces carry the same local run.id.
• --resume from that run's configs/resolved/eval.json loads and reports nothing to resume. The same file with "push": true added fails with "Extra inputs are not permitted".

🤖 Generated with Claude Code

Note

Remove platform push from vf-eval and drop the prime-runs dependency

• Deletes the platform run lifecycle in platform.py: open_run, log_episodes, finish_run, abort_run, PushState, and related helpers are gone. run_eval in runner.py still writes completed episodes to the local trace store.
• Removes the push field from EvalConfig and the attach field and assign_id method from RunConfig. Each run now gets a UUID run ID at construction, so reading id no longer waits for a platform run to open.
• The dashboard in eval.py no longer takes push state or renders a push-progress footer.
• Drops prime-runs from pyproject.toml and removes --no-push from docs and the example integration test. Deletes test_platform.py.
• Behavioral Change: RunConfig.id is a locally generated UUID instead of a platform-assigned ID, and VF_RUN_ID is set from config.run.id. Any code reading EvalConfig.push, RunConfig.attach, verifiers.v1.utils.platform.build_samples, or passing --no-push will break; in-tree usages are updated.

^{Macroscope summarized ab63017.}

---

Note

Medium Risk
Breaking CLI/config surface (push, attach, saved eval JSON) and run ID semantics change from platform-assigned to local UUID; core eval path remains disk-backed traces only.

Overview
vf-eval is local-only for traces: it no longer opens, streams to, finishes, or aborts a Prime Intellect platform run. Episodes still append to traces.jsonl as before; platform uploads are expected via prime-rl's eval.

Removed verifiers/v1/utils/platform.py (including PushState, open_run, log_episodes, finish_run, abort_run) and its unit tests. prime-runs is dropped from pyproject.toml and uv.lock; prime-traces is removed from exclude-newer-package only.

Config breaking changes: EvalConfig.push (--push / --no-push), RunConfig.attach, and assign_id are gone. run.id is a uuid4().hex at config construction (still on traces and $VF_RUN_ID). Saved resolved configs that still contain "push": true will fail validation on resume/replay until that key is removed.

Runner and dashboard: run_eval no longer calls platform helpers or shielded close-out; the live dashboard no longer shows push progress URLs. Docs and integration tests drop --no-push from example commands.

^{Reviewed by Cursor Bugbot for commit ab63017. Bugbot is set up for automated code reviews on this repo. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — Evaluations become local-only, and existing push/attach flags and saved configs lose compatibility. Human judgment is needed on the migration and release plan, including preventing hosted deployments from selecting the removed attachment path.

^{You can add or adjust custom eligibility rules. Learn more.}