feat!: remove the serve CLI, rename ServingConfig to ServeConfig - #2237
Merged
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mikasenghaas
added a commit
to PrimeIntellect-ai/prime-rl
that referenced
this pull request
Aug 3, 2026
Bumps deps/verifiers to PrimeIntellect-ai/verifiers#2237 (serve CLI removed, ServingConfig renamed ServeConfig); prl's source-level ServingConfig follows suit Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mikasenghaas
marked this pull request as ready for review
August 3, 2026 23:06
Contributor
ApprovabilityVerdict: Needs human review This PR removes the You can customize Macroscope's approvability policy. Learn more. |
hallerite
approved these changes
Aug 3, 2026
mikasenghaas
added a commit
to PrimeIntellect-ai/prime-rl
that referenced
this pull request
Aug 4, 2026
* feat!: remove env-server sidecaring, launcher spawns at deterministic addresses The orchestrator never spawns env servers anymore — it always connects to each source's serve.address and polls until the server is up. Addresses are assigned deterministically at config validation (tcp://127.0.0.1:5000+ across train then eval sources), so the launcher (local rl + multinode sbatch template) spawns the servers in parallel with the orchestrator and both sides agree on where each server lives from the config alone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: docstring + test tidy Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor!: derive env-server addresses, require address on env-server config Addresses are wiring, not intent — like the transport blocks, they are no longer configurable on sources. The orchestrator derives each server's address from the source's position in the config; the env-server entrypoint takes a required address the launcher writes per source. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: env-server config mirrors verifiers' serve CLI, per-run base port EnvServerConfig composes vf's env/serve/legacy blocks verbatim, so an env-server TOML reads like a vf serve config. orchestrator.env_server_base_port offsets the derived port range, giving concurrent runs on one host (e.g. multi-run orchestrators) each their own 1:1 env server. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(configs): multi-env debug config (2 reverse-text train sources + eval) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: rename env server entrypoint to `env` (keep `env-server` alias), env_server_addresses -> env_addresses Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: env/ prefix for env server stop-event keys Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: rename env server naming to env throughout EnvServerConfig -> configs/env.py:EnvConfig, source-level EnvConfig -> SourceConfig, entrypoint moves to entrypoints/env.py, env_server_base_port -> env_base_port, proctitle PRIME-RL::Env Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: trim env entrypoint row Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: drop the verifiers serve CLI, adopt vf.ServeConfig Bumps deps/verifiers to PrimeIntellect-ai/verifiers#2237 (serve CLI removed, ServingConfig renamed ServeConfig); prl's source-level ServingConfig follows suit Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: drop the removed serve CLI reference from EnvConfig Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: pin verifiers to main (serve CLI removal merged) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci: slim install imports configs.env Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * revert: keep env-server naming, validate env identity on EnvServerConfig The env console script shadows coreutils env in activated shells — back to env-server as the sole entrypoint (still at entrypoints/, orchestrator source class stays EnvConfig as on main). EnvServerConfig regains the identity validation the nested source config used to provide (missing env, mixed v0/v1) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(slurm): remove apostrophe from env-server launch comment The multi-node srun body is one single-quoted bash -c string, so the apostrophe in "source's" closed it early: every rendered sbatch failed to parse with a syntax error near unexpected token '(' and the job died before any component started. Reword the comment and note the constraint. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
hallerite
added a commit
that referenced
this pull request
Sep 2, 2026
Pure deletions; no behaviour change. - configs/eval, configs/gepa, configs/rl, configs/zero3.yaml, configs/endpoints.toml: v0 `vf-eval` / training formats, unreferenced since the v0 stack was removed; drop the test filter that skipped them - pyproject: inert nemo-gym dependency-metadata (not a dependency since #2476), pytest markers no test applies, cellpylib warning filter - Harness.run: superseded by HarnessSession.turn, zero callers - Agents.__iter__/__len__: no reader in verifiers, prime-rl, or the env repos - utils.git.snapshot_untracked: never called - cli.output.saved_config_path: drop the pre-#2429 configs/<cli>.json fallback; eval/gepa usage gate: drop the --taskset./--harness. prefixes of flags removed in #2157/#2237 - mcp: ServerBase.EXTRAS and RUNTIME_PYTHON hooks that no server declares - prime_agent: SUPPORTS_RESUME is never read on an ACP harness Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
xeophon
pushed a commit
that referenced
this pull request
Sep 2, 2026
Pure deletions. Every item has zero references in verifiers, prime-rl, prime-envs and community-environments (grep + GitHub code search); nothing here changes behaviour. **v0 config trees (−820 lines).** `configs/eval/`, `configs/gepa/`, `configs/rl/`, `configs/zero3.yaml`, `configs/endpoints.toml` are the `vf-eval` / DeepSpeed formats of the removed v0 stack. `tests/v1/test_configs.py` already excluded them; the filter goes with them. **pyproject.** The `[[tool.uv.dependency-metadata]] nemo-gym` block is inert (nemo-gym stopped being a dependency in #2476; the PEP 723 header in `tasksets/nemo_gym/server.py` is the one that is used), so `uv.lock` loses its manifest entry. Markers `integration`, `slow`, `unit`, `parsers`, `rubrics`, `environments` are applied by no test; the `cellpylib` warning filter targets a package that is not in the lock. **Dead code.** - `Harness.run()`: the pre-session launch/resume dispatch, superseded by `HarnessSession.turn`. - `Agents.__iter__`/`__len__`: no reader anywhere; every env addresses agents by attribute. - `saved_config_path`: the fallback to the pre-#2429 `configs/<cli>.json` location. Only `configs/resolved/` is written. - `eval`/`gepa` usage gate: the `--taskset.`/`--harness.` prefixes kept alive for flags removed in #2157/#2237. - `ServerBase.EXTRAS` and the `RUNTIME_PYTHON` hook in `mcp/launch.py`: declared by no server anywhere. - `PrimeAgentHarness.SUPPORTS_RESUME`: `ACPHarness` overrides `session()`, so both sites that read the flag short-circuit before reaching it. `snapshot_untracked` was removed in the first revision and restored in the second: prime-envs #792 calls it from four SWE tasksets (`capture_patch(ignore=...)`). Open PRs in the env repos are now part of the check. Deliberately not in this PR: anything on the `Trace`/`Episode` record model (`Episode` aggregates, `PolicySpan.drift`, `Trace.last_message`) — those have downstream consumers outside this repo; `agent_config_fields` (its `_declared_agent_configs` twin checks field defaults, not values, so swapping is not a pure deletion), `WireTrace`, `InterceptionError`, the `Branch` trainer-side properties, and every docs/docstring wording fix. Those are separate PRs. Checks run on the branch: `ruff check`, `ruff format --check`, `ty check verifiers`, and `pytest tests/v1 -m "not e2e"` (82 passed). 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Mostly deletions of unreferenced configs and code; the only behavioral tweaks are stricter CLI axis detection, resolved-config-only replay paths, and always running full MCP sandbox installs. > > **Overview** > Removes **v0-era configuration** that is no longer part of the v1 stack: the entire `configs/eval/`, `configs/gepa/`, `configs/rl/` trees, `configs/endpoints.toml`, and `configs/zero3.yaml`. Config validation in `tests/v1/test_configs.py` no longer special-cases `endpoints.toml`. > > **Tooling cleanup** drops the unused `nemo-gym` `[[tool.uv.dependency-metadata]]` block from `pyproject.toml` (and the matching `uv.lock` manifest), unused pytest markers, and a `cellpylib` warning filter for a package not in the lock. > > **Dead v1 code paths** are deleted without replacing callers: `Harness.run()` (segment dispatch now lives on `HarnessSession.turn`), `Agents.__iter__`/`__len__`, legacy resolved-config lookup in `saved_config_path`, narrowed `eval`/`gepa` CLI usage gates (no `--taskset.` / `--harness.` shortcuts), MCP sandbox install no longer honors `ServerBase.EXTRAS` or `RUNTIME_PYTHON`, and `PrimeAgentHarness.SUPPORTS_RESUME` (unused because ACP harnesses use `session()`). > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 8389294. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- Macroscope's pull request summary starts here --> <!-- Macroscope will only edit the content between these invisible markers, and the markers themselves will not be visible in the GitHub rendered markdown. --> <!-- If you delete either of the start / end markers from your PR's description, Macroscope will append its summary at the bottom of the description. --> > [!NOTE] > ### Remove v0 config files, dead code, and fix CLI usage-gate bypass in v1 > - Deletes leftover v0 configs under `configs/` (endpoint registry, eval/gepa/rl TOMLs, `zero3.yaml`) and removes unused pytest markers, dependency metadata, and dead class members (`Agents.__iter__`, `Agents.__len__`, `ServerBase.EXTRAS`, `PrimeAgentHarness.SUPPORTS_RESUME`). > - Restricts the typed-axis usage-gate bypass in [main.py](https://github.com/PrimeIntellect-ai/verifiers/pull/2496/files#diff-1335625d1379d7f1a4bf6a2f961d4b5fde5f3a9bfb8eb4cc736859f08698a8c8) and [gepa.py](https://github.com/PrimeIntellect-ai/verifiers/pull/2496/files#diff-dc87040f272d5f7871025c1cd9da91e56670d1cf1deff808037cb5bc356376bc) to `--env.`/`--serve.` prefixes only; `--taskset.` and `--harness.` arguments no longer suppress the usage message. > - Removes the `configs/` fallback in [output.py](https://github.com/PrimeIntellect-ai/verifiers/pull/2496/files#diff-07fce45dd0fa633f34d25e19962ab801b705a509774833de0d81bfe872787e8b): `saved_config_path` now searches only `configs/resolved/`. > - Changes non-subprocess MCP server launch in [launch.py](https://github.com/PrimeIntellect-ai/verifiers/pull/2496/files#diff-989d87123d40d48511f9e1df74594d2e30776e9a99c11eca49d9cc20bf9e2230) to always install a sandbox venv and use its Python, dropping the `RUNTIME_PYTHON` override and server-class extras suffix. > - Risk: non-subprocess MCP servers no longer honor a prebuilt `RUNTIME_PYTHON` executable; `saved_config_path` returns `None` for runs with JSON only under `configs/` rather than `configs/resolved/`. > > <!-- Macroscope's review summary starts here --> > > <sup><a href="https://app.macroscope.com">Macroscope</a> summarized 8389294.</sup> > <!-- Macroscope's review summary ends here --> > <!-- Macroscope's pull request summary ends here --> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Removes the
serveCLI entrypoint: hosting an env server from the command line is the trainer's job (prime-rl'suv run env), and nothing in verifiers, prime-rl, or research-environments invokesuv run serve— the server-backed eval path already callsserve_envin-process.verifiers/v1/cli/serve.py, its run configverifiers/v1/configs/cli/serve.py, and theserveconsole script.ServingConfig→ServeConfig(the name the CLI config freed up): the reusable[serve]block — pool, address, max_concurrent — read by whoever hosts an env.serve_env,pool_serve_kwargs, and the eval CLI's[serve]block are unchanged apart from the type name.Breaking
uv run serveis gone. Host an env server programmatically viaserve_env, or through a trainer (prime-rl'suv run env, one server per source).vf.ServingConfigis renamed tovf.ServeConfig(same fields:pool,address,max_concurrent). The CLI-onlyServeConfig(env + serve + legacy + verbose/dry_run) is removed with the CLI; it was not exported at theverifiers.v1top level.Companion: PrimeIntellect-ai/prime-rl#3162
🤖 Generated with Claude Code
Note
Remove the
serveCLI and renameServingConfigtoServeConfigserveconsole script entry point from pyproject.toml; theservecommand can no longer be invoked via the package.ServingConfigtoServeConfigin verifiers/v1/configs/serve.py and updates all imports and__all__exports across the package.ServeConfigfromverifiers.v1.configs.clisince the CLI config module forserveis also deleted.ServingConfigor invoking theserveCLI entry point will break.Macroscope summarized d7b01c9.
Note
Medium Risk
Breaking API and CLI removal affects any external scripts using
uv run serveor importingServingConfig; in-repo eval/server paths only need the rename.Overview
Breaking: Removes the standalone
serveconsole script and the entireuv run serveCLI (verifiers/v1/cli/serve.pyplusverifiers/v1/configs/cli/serve.py). Hosting an env server is expected viaserve_envin-process (e.g. eval--server) or through trainers such as prime-rl’suv run env, not a dedicated verifiers CLI.Renames the shared
[serve]pydantic block fromServingConfigtoServeConfiginverifiers/v1/configs/serve.py, with matching import/__all__updates (includingEvalConfig.serve). Field semantics (pool,address,max_concurrent) and helpers likepool_serve_kwargs/serve_envare unchanged aside from the type name. Docs inpyproject.tomlnow listvalidateinstead ofserveamong default v1 CLIs.Callers must stop using
uv run serveand replaceServingConfigimports withServeConfig.Reviewed by Cursor Bugbot for commit d7b01c9. Bugbot is set up for automated code reviews on this repo. Configure here.