Skip to content

fix(v1,serve): env-worker memory — step slimming, registry eviction, intercept discard, worker trim #1608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hubert-marek merged 2 commits into from
Jun 10, 2026
Merged

fix(v1,serve): env-worker memory — step slimming, registry eviction, intercept discard, worker trim #1608

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3c14162
fix(v1): drop response message dump from trajectory steps
hubert-marek Jun 10, 2026
c8e2b90
fix(v1): evict completed rollouts from the live-trajectory registry
hubert-marek Jun 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion verifiers/v1/runtime.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -875,10 +875,21 @@ async def submit_model_request(
is_truncated = response.message.is_truncated or (
tokens is not None and bool(tokens.get("is_truncated"))
)
# Identity/usage only: the full Response dump would re-store the
# message content (= ``completion``) and ``message.tokens``
# (= ``tokens``, incl. per-token attribution), doubling every
# trajectory step in worker memory, on the wire, and in the
# orchestrator's group buffers. v1 step readers that want the
# heavy fields isinstance-check the live ``Response``, which a
# serialized dict never passes; usage is recorded on state at
# this call site via ``record_response_usage``.
response_meta = serializable(response)
if isinstance(response_meta, dict):
response_meta.pop("message", None)

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep raw response tokens available for renderer bridging

When renderer rollouts hit training-budget truncation, renderer_client._step_token_ids intentionally falls back from empty step["tokens"].completion_ids to step["response"]["message"]["tokens"] (see verifiers/clients/renderer_client.py and the fallback tests in tests/test_renderer_client.py). Removing message here makes that fallback return None, so subsequent turns are fully re-rendered instead of bridged and can lose the cache/multimodal placeholder continuity the renderer path relies on for long multimodal rollouts. If the goal is to drop duplicated content, keep a minimal raw-token/is-truncated sidecar rather than deleting the whole message.

Useful? React with 👍 / 👎.

step = {
"prompt": serializable(prompt),
"completion": serializable(completion),
"response": serializable(response),
"response": response_meta,
"tokens": serializable(tokens),
"reward": None,
"advantage": None,
Expand Down
Loading