Skip to content

fix(v1,serve): env-worker memory — step slimming, registry eviction, intercept discard, worker trim #1608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hubert-marek merged 2 commits into from
Jun 10, 2026
Merged

fix(v1,serve): env-worker memory — step slimming, registry eviction, intercept discard, worker trim #1608

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3c14162
fix(v1): drop response message dump from trajectory steps
hubert-marek Jun 10, 2026
c8e2b90
fix(v1): evict completed rollouts from the live-trajectory registry
hubert-marek Jun 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion verifiers/v1/runtime.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -875,10 +875,21 @@ async def submit_model_request(
is_truncated = response.message.is_truncated or (
tokens is not None and bool(tokens.get("is_truncated"))
)
# Identity/usage only: the full Response dump would re-store the
# message content (= ``completion``) and ``message.tokens``
# (= ``tokens``, incl. per-token attribution), doubling every
# trajectory step in worker memory, on the wire, and in the
# orchestrator's group buffers. v1 step readers that want the
# heavy fields isinstance-check the live ``Response``, which a
# serialized dict never passes; usage is recorded on state at
# this call site via ``record_response_usage``.
response_meta = serializable(response)
if isinstance(response_meta, dict):
response_meta.pop("message", None)

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep raw response tokens available for renderer bridging

When renderer rollouts hit training-budget truncation, renderer_client._step_token_ids intentionally falls back from empty step["tokens"].completion_ids to step["response"]["message"]["tokens"] (see verifiers/clients/renderer_client.py and the fallback tests in tests/test_renderer_client.py). Removing message here makes that fallback return None, so subsequent turns are fully re-rendered instead of bridged and can lose the cache/multimodal placeholder continuity the renderer path relies on for long multimodal rollouts. If the goal is to drop duplicated content, keep a minimal raw-token/is-truncated sidecar rather than deleting the whole message.

Useful? React with 👍 / 👎.

step = {
"prompt": serializable(prompt),
"completion": serializable(completion),
"response": serializable(response),
"response": response_meta,
"tokens": serializable(tokens),
"reward": None,
"advantage": None,
Expand Down Expand Up @@ -978,6 +989,12 @@ async def cleanup_rollout(self, task: Task, state: State) -> None:
await self.close_mcp_tools(state)
self.release_scoped_tools("rollout", state)
await self.release_model_client(state)
# The live-trajectory registry (register_trajectory) is only read by
# resolve_trajectory for handle-borrowing sub-runtime states, whose
# lifetime is within the owning rollout — without this pop the
# long-lived Runtime retains every completed rollout's full
# trajectory and the env worker leaks ~50-400MB per rollout.
self.trajectories.pop(str(state["trajectory_id"]), None)

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Defer trajectory eviction until group cleanup

For grouped rollouts this cleanup runs before the group phase: Env._run_group_states awaits all harness.run(...) calls and only then calls harness.score_group(...), while Harness.run skips cleanup_group when group_key is present. Popping the live trajectory here therefore makes any group update/reward/cleanup that runs a stored child state with transcript="append" fail in resolve_trajectory with No live trajectory registered..., even though those runtime handles are intentionally retained until group cleanup. This eviction should be delayed for grouped states (or repeated in cleanup_group) so group-stage borrowers can still append to the parent trajectory.

Useful? React with 👍 / 👎.

self.release_tool_handles(state)

async def cleanup_group(self, tasks: list[Task], states: list[State]) -> None:
Expand Down
Loading