Skip to content

fix: build a chat-completions client for v0 eval in the legacy bridge - #1615

Merged
mikasenghaas merged 1 commit into
feat/nano-as-v1from
fix/legacy-bridge-eval-client
Jun 10, 2026
Merged

fix: build a chat-completions client for v0 eval in the legacy bridge#1615
mikasenghaas merged 1 commit into
feat/nano-as-v1from
fix/legacy-bridge-eval-client

Conversation

@mikasenghaas

@mikasenghaas mikasenghaas commented Jun 10, 2026

Copy link
Copy Markdown
Member

Summary

Fixes v0 env eval crashing in the legacy bridge.

The orchestrator drives both training and eval rollouts of a v0 env through the same LegacyEnvServer (run_rollout / run_group_run_v0_v0_client). Training uses a renderer (token-in/out) client; eval uses an OpenAI chat-completions client (pool.get_eval_client()). The guard added in #1613 raised on any non-renderer config:

ValueError: the v0 legacy bridge trains through a renderer (token-in/out) client,
got client type 'openai'; MITO (chat-completions) training of v0 envs is not supported

so it broke v0 eval.

_v0_client now dispatches on the config type instead of raising:

  • RendererClientConfig (type="renderers", training) → v0 renderer client (unchanged).
  • OpenAIClientConfig (type="openai", eval) → v0 chat-completions client.

Verification

  • ruff check --isolated / ruff format --isolated --check clean.
  • _v0_client(OpenAIClientConfig(), model) builds an OpenAIChatCompletionsClient (no longer raises); a RendererClientConfig builds the renderer client as before.

Note

Support OpenAI chat-completions client config in LegacyEnvServer._v0_client

Previously, _v0_client in legacy.py raised a ValueError for any non-renderer client config. It now accepts OpenAI chat-completions configs and builds a V0ClientConfig with client_type="openai_chat_completions", omitting renderer-specific fields. Renderer configs continue to pin the tokenizer to renderer_model_name as before.

Macroscope summarized 06b45e2.


Note

Low Risk
Narrow adapter change in the legacy bridge; renderer training path is unchanged and eval now mirrors existing _eval_client behavior.

Overview
Fixes v0 environment eval when rollouts go through LegacyEnvServer (orchestrator eval uses OpenAIClientConfig, not the renderer used for training).

LegacyEnvServer._v0_client no longer raises on non-renderer v1 configs. It branches on config type: RendererClientConfig still maps to a v0 renderer client (tokenizer pinned via renderer_model_name); any other config (eval’s OpenAI chat-completions) maps to v0 openai_chat_completions with base URL, API key var, and headers only. Client caching is unchanged.

Training-only rejection of chat-completions for v0 MITO is removed in favor of supporting eval while keeping the renderer path for training.

Reviewed by Cursor Bugbot for commit 06b45e2. Bugbot is set up for automated code reviews on this repo. Configure here.

The orchestrator drives eval rollouts of v0 envs through the same
LegacyEnvServer (run_rollout/run_group -> _run_v0 -> _v0_client), and eval
uses an OpenAI chat-completions client. The #1613 guard raised on any
non-renderer config, so it broke v0 eval ("MITO ... not supported").

Dispatch on the config type instead: a renderer config (training, TITO)
builds a v0 renderer client; an OpenAI config (eval) builds a v0
chat-completions client.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mikasenghaas
mikasenghaas marked this pull request as ready for review June 10, 2026 23:08
@mikasenghaas
mikasenghaas merged commit a9427e2 into feat/nano-as-v1 Jun 10, 2026
3 checks passed
@macroscopeapp

macroscopeapp Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR removes an explicit error guard and enables a previously-blocked code path for chat-completions clients in the v0 legacy bridge. While the change is small, it enables new runtime behavior rather than fixing broken functionality, warranting human review.

You can customize Macroscope's approvability policy. Learn more.

pull Bot pushed a commit to Stars1233/verifiers that referenced this pull request Jun 23, 2026
…PrimeIntellect-ai#1615)

The orchestrator drives eval rollouts of v0 envs through the same
LegacyEnvServer (run_rollout/run_group -> _run_v0 -> _v0_client), and eval
uses an OpenAI chat-completions client. The PrimeIntellect-ai#1613 guard raised on any
non-renderer config, so it broke v0 eval ("MITO ... not supported").

Dispatch on the config type instead: a renderer config (training, TITO)
builds a v0 renderer client; an OpenAI config (eval) builds a v0
chat-completions client.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant