Skip to content

feat!: remove the serve CLI, rename ServingConfig to ServeConfig - #2237

Merged
hallerite merged 1 commit into
mainfrom
feat/remove-serve-cli
Aug 3, 2026
Merged

feat!: remove the serve CLI, rename ServingConfig to ServeConfig#2237
hallerite merged 1 commit into
mainfrom
feat/remove-serve-cli

Conversation

@mikasenghaas

@mikasenghaas mikasenghaas commented Aug 3, 2026

Copy link
Copy Markdown
Member

Summary

Removes the serve CLI entrypoint: hosting an env server from the command line is the trainer's job (prime-rl's uv run env), and nothing in verifiers, prime-rl, or research-environments invokes uv run serve — the server-backed eval path already calls serve_env in-process.

  • Deletes verifiers/v1/cli/serve.py, its run config verifiers/v1/configs/cli/serve.py, and the serve console script.
  • Renames ServingConfigServeConfig (the name the CLI config freed up): the reusable [serve] block — pool, address, max_concurrent — read by whoever hosts an env. serve_env, pool_serve_kwargs, and the eval CLI's [serve] block are unchanged apart from the type name.

Breaking

  • uv run serve is gone. Host an env server programmatically via serve_env, or through a trainer (prime-rl's uv run env, one server per source).
  • vf.ServingConfig is renamed to vf.ServeConfig (same fields: pool, address, max_concurrent). The CLI-only ServeConfig (env + serve + legacy + verbose/dry_run) is removed with the CLI; it was not exported at the verifiers.v1 top level.

Companion: PrimeIntellect-ai/prime-rl#3162

🤖 Generated with Claude Code

Note

Remove the serve CLI and rename ServingConfig to ServeConfig

  • Deletes verifiers/v1/cli/serve.py and removes the serve console script entry point from pyproject.toml; the serve command can no longer be invoked via the package.
  • Renames ServingConfig to ServeConfig in verifiers/v1/configs/serve.py and updates all imports and __all__ exports across the package.
  • Removes ServeConfig from verifiers.v1.configs.cli since the CLI config module for serve is also deleted.
  • Risk: breaking change — any code importing ServingConfig or invoking the serve CLI entry point will break.

Macroscope summarized d7b01c9.


Note

Medium Risk
Breaking API and CLI removal affects any external scripts using uv run serve or importing ServingConfig; in-repo eval/server paths only need the rename.

Overview
Breaking: Removes the standalone serve console script and the entire uv run serve CLI (verifiers/v1/cli/serve.py plus verifiers/v1/configs/cli/serve.py). Hosting an env server is expected via serve_env in-process (e.g. eval --server) or through trainers such as prime-rl’s uv run env, not a dedicated verifiers CLI.

Renames the shared [serve] pydantic block from ServingConfig to ServeConfig in verifiers/v1/configs/serve.py, with matching import/__all__ updates (including EvalConfig.serve). Field semantics (pool, address, max_concurrent) and helpers like pool_serve_kwargs / serve_env are unchanged aside from the type name. Docs in pyproject.toml now list validate instead of serve among default v1 CLIs.

Callers must stop using uv run serve and replace ServingConfig imports with ServeConfig.

Reviewed by Cursor Bugbot for commit d7b01c9. Bugbot is set up for automated code reviews on this repo. Configure here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mikasenghaas added a commit to PrimeIntellect-ai/prime-rl that referenced this pull request Aug 3, 2026
Bumps deps/verifiers to PrimeIntellect-ai/verifiers#2237 (serve CLI removed,
ServingConfig renamed ServeConfig); prl's source-level ServingConfig follows suit

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mikasenghaas
mikasenghaas requested a review from hallerite August 3, 2026 23:05
@mikasenghaas
mikasenghaas marked this pull request as ready for review August 3, 2026 23:06
@macroscopeapp

macroscopeapp Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR removes the serve CLI command entirely, which is a breaking change to the public API. While the rename from ServingConfig to ServeConfig is mechanical, removing a public CLI entrypoint warrants human review to confirm the intended deprecation path.

You can customize Macroscope's approvability policy. Learn more.

@hallerite
hallerite merged commit d30a3f4 into main Aug 3, 2026
13 checks passed
@hallerite
hallerite deleted the feat/remove-serve-cli branch August 3, 2026 23:32
mikasenghaas added a commit to PrimeIntellect-ai/prime-rl that referenced this pull request Aug 4, 2026
* feat!: remove env-server sidecaring, launcher spawns at deterministic addresses

The orchestrator never spawns env servers anymore — it always connects to
each source's serve.address and polls until the server is up. Addresses
are assigned deterministically at config validation (tcp://127.0.0.1:5000+
across train then eval sources), so the launcher (local rl + multinode
sbatch template) spawns the servers in parallel with the orchestrator and
both sides agree on where each server lives from the config alone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: docstring + test tidy

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor!: derive env-server addresses, require address on env-server config

Addresses are wiring, not intent — like the transport blocks, they are no
longer configurable on sources. The orchestrator derives each server's
address from the source's position in the config; the env-server entrypoint
takes a required address the launcher writes per source.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: env-server config mirrors verifiers' serve CLI, per-run base port

EnvServerConfig composes vf's env/serve/legacy blocks verbatim, so an
env-server TOML reads like a vf serve config. orchestrator.env_server_base_port
offsets the derived port range, giving concurrent runs on one host (e.g.
multi-run orchestrators) each their own 1:1 env server.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(configs): multi-env debug config (2 reverse-text train sources + eval)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: rename env server entrypoint to `env` (keep `env-server` alias), env_server_addresses -> env_addresses

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: env/ prefix for env server stop-event keys

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: rename env server naming to env throughout

EnvServerConfig -> configs/env.py:EnvConfig, source-level EnvConfig -> SourceConfig,
entrypoint moves to entrypoints/env.py, env_server_base_port -> env_base_port,
proctitle PRIME-RL::Env

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: trim env entrypoint row

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: drop the verifiers serve CLI, adopt vf.ServeConfig

Bumps deps/verifiers to PrimeIntellect-ai/verifiers#2237 (serve CLI removed,
ServingConfig renamed ServeConfig); prl's source-level ServingConfig follows suit

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: drop the removed serve CLI reference from EnvConfig

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: pin verifiers to main (serve CLI removal merged)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: slim install imports configs.env

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* revert: keep env-server naming, validate env identity on EnvServerConfig

The env console script shadows coreutils env in activated shells — back to
env-server as the sole entrypoint (still at entrypoints/, orchestrator source
class stays EnvConfig as on main). EnvServerConfig regains the identity
validation the nested source config used to provide (missing env, mixed v0/v1)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(slurm): remove apostrophe from env-server launch comment

The multi-node srun body is one single-quoted bash -c string, so the
apostrophe in "source's" closed it early: every rendered sbatch failed to
parse with a syntax error near unexpected token '(' and the job died before
any component started. Reword the comment and note the constraint.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
hallerite added a commit that referenced this pull request Sep 2, 2026
Pure deletions; no behaviour change.

- configs/eval, configs/gepa, configs/rl, configs/zero3.yaml,
  configs/endpoints.toml: v0 `vf-eval` / training formats, unreferenced
  since the v0 stack was removed; drop the test filter that skipped them
- pyproject: inert nemo-gym dependency-metadata (not a dependency since
  #2476), pytest markers no test applies, cellpylib warning filter
- Harness.run: superseded by HarnessSession.turn, zero callers
- Agents.__iter__/__len__: no reader in verifiers, prime-rl, or the env repos
- utils.git.snapshot_untracked: never called
- cli.output.saved_config_path: drop the pre-#2429 configs/<cli>.json
  fallback; eval/gepa usage gate: drop the --taskset./--harness. prefixes
  of flags removed in #2157/#2237
- mcp: ServerBase.EXTRAS and RUNTIME_PYTHON hooks that no server declares
- prime_agent: SUPPORTS_RESUME is never read on an ACP harness

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
xeophon pushed a commit that referenced this pull request Sep 2, 2026
Pure deletions. Every item has zero references in verifiers, prime-rl,
prime-envs and community-environments (grep + GitHub code search);
nothing here changes behaviour.

**v0 config trees (−820 lines).** `configs/eval/`, `configs/gepa/`,
`configs/rl/`, `configs/zero3.yaml`, `configs/endpoints.toml` are the
`vf-eval` / DeepSpeed formats of the removed v0 stack.
`tests/v1/test_configs.py` already excluded them; the filter goes with
them.

**pyproject.** The `[[tool.uv.dependency-metadata]] nemo-gym` block is
inert (nemo-gym stopped being a dependency in #2476; the PEP 723 header
in `tasksets/nemo_gym/server.py` is the one that is used), so `uv.lock`
loses its manifest entry. Markers `integration`, `slow`, `unit`,
`parsers`, `rubrics`, `environments` are applied by no test; the
`cellpylib` warning filter targets a package that is not in the lock.

**Dead code.**
- `Harness.run()`: the pre-session launch/resume dispatch, superseded by
`HarnessSession.turn`.
- `Agents.__iter__`/`__len__`: no reader anywhere; every env addresses
agents by attribute.
- `saved_config_path`: the fallback to the pre-#2429
`configs/<cli>.json` location. Only `configs/resolved/` is written.
- `eval`/`gepa` usage gate: the `--taskset.`/`--harness.` prefixes kept
alive for flags removed in #2157/#2237.
- `ServerBase.EXTRAS` and the `RUNTIME_PYTHON` hook in `mcp/launch.py`:
declared by no server anywhere.
- `PrimeAgentHarness.SUPPORTS_RESUME`: `ACPHarness` overrides
`session()`, so both sites that read the flag short-circuit before
reaching it.

`snapshot_untracked` was removed in the first revision and restored in
the second: prime-envs #792 calls it from four SWE tasksets
(`capture_patch(ignore=...)`). Open PRs in the env repos are now part of
the check.

Deliberately not in this PR: anything on the `Trace`/`Episode` record
model (`Episode` aggregates, `PolicySpan.drift`, `Trace.last_message`) —
those have downstream consumers outside this repo; `agent_config_fields`
(its `_declared_agent_configs` twin checks field defaults, not values,
so swapping is not a pure deletion), `WireTrace`, `InterceptionError`,
the `Branch` trainer-side properties, and every docs/docstring wording
fix. Those are separate PRs.

Checks run on the branch: `ruff check`, `ruff format --check`, `ty check
verifiers`, and `pytest tests/v1 -m "not e2e"` (82 passed).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Mostly deletions of unreferenced configs and code; the only behavioral
tweaks are stricter CLI axis detection, resolved-config-only replay
paths, and always running full MCP sandbox installs.
> 
> **Overview**
> Removes **v0-era configuration** that is no longer part of the v1
stack: the entire `configs/eval/`, `configs/gepa/`, `configs/rl/` trees,
`configs/endpoints.toml`, and `configs/zero3.yaml`. Config validation in
`tests/v1/test_configs.py` no longer special-cases `endpoints.toml`.
> 
> **Tooling cleanup** drops the unused `nemo-gym`
`[[tool.uv.dependency-metadata]]` block from `pyproject.toml` (and the
matching `uv.lock` manifest), unused pytest markers, and a `cellpylib`
warning filter for a package not in the lock.
> 
> **Dead v1 code paths** are deleted without replacing callers:
`Harness.run()` (segment dispatch now lives on `HarnessSession.turn`),
`Agents.__iter__`/`__len__`, legacy resolved-config lookup in
`saved_config_path`, narrowed `eval`/`gepa` CLI usage gates (no
`--taskset.` / `--harness.` shortcuts), MCP sandbox install no longer
honors `ServerBase.EXTRAS` or `RUNTIME_PYTHON`, and
`PrimeAgentHarness.SUPPORTS_RESUME` (unused because ACP harnesses use
`session()`).
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
8389294. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

<!-- Macroscope's pull request summary starts here -->
<!-- Macroscope will only edit the content between these invisible
markers, and the markers themselves will not be visible in the GitHub
rendered markdown. -->
<!-- If you delete either of the start / end markers from your PR's
description, Macroscope will append its summary at the bottom of the
description. -->
> [!NOTE]
> ### Remove v0 config files, dead code, and fix CLI usage-gate bypass
in v1
> - Deletes leftover v0 configs under `configs/` (endpoint registry,
eval/gepa/rl TOMLs, `zero3.yaml`) and removes unused pytest markers,
dependency metadata, and dead class members (`Agents.__iter__`,
`Agents.__len__`, `ServerBase.EXTRAS`,
`PrimeAgentHarness.SUPPORTS_RESUME`).
> - Restricts the typed-axis usage-gate bypass in
[main.py](https://github.com/PrimeIntellect-ai/verifiers/pull/2496/files#diff-1335625d1379d7f1a4bf6a2f961d4b5fde5f3a9bfb8eb4cc736859f08698a8c8)
and
[gepa.py](https://github.com/PrimeIntellect-ai/verifiers/pull/2496/files#diff-dc87040f272d5f7871025c1cd9da91e56670d1cf1deff808037cb5bc356376bc)
to `--env.`/`--serve.` prefixes only; `--taskset.` and `--harness.`
arguments no longer suppress the usage message.
> - Removes the `configs/` fallback in
[output.py](https://github.com/PrimeIntellect-ai/verifiers/pull/2496/files#diff-07fce45dd0fa633f34d25e19962ab801b705a509774833de0d81bfe872787e8b):
`saved_config_path` now searches only `configs/resolved/`.
> - Changes non-subprocess MCP server launch in
[launch.py](https://github.com/PrimeIntellect-ai/verifiers/pull/2496/files#diff-989d87123d40d48511f9e1df74594d2e30776e9a99c11eca49d9cc20bf9e2230)
to always install a sandbox venv and use its Python, dropping the
`RUNTIME_PYTHON` override and server-class extras suffix.
> - Risk: non-subprocess MCP servers no longer honor a prebuilt
`RUNTIME_PYTHON` executable; `saved_config_path` returns `None` for runs
with JSON only under `configs/` rather than `configs/resolved/`.
>
> <!-- Macroscope's review summary starts here -->
>
> <sup><a href="https://app.macroscope.com">Macroscope</a> summarized
8389294.</sup>
> <!-- Macroscope's review summary ends here -->
>
<!-- Macroscope's pull request summary ends here -->

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants