Skip to content

fix: carry v0 truncation flag through legacy bridge + declare compact's verifiers dep - #1824

Merged
mikasenghaas merged 3 commits into
feat/nano-as-v1from
fix/v1-legacy-truncation-compact-dep
Jun 22, 2026
Merged

fix: carry v0 truncation flag through legacy bridge + declare compact's verifiers dep#1824
mikasenghaas merged 3 commits into
feat/nano-as-v1from
fix/v1-legacy-truncation-compact-dep

Conversation

@mikasenghaas

@mikasenghaas mikasenghaas commented Jun 22, 2026

Copy link
Copy Markdown
Member

Summary

Two fixes to the v1 surface, both from review of the nano-as-v1 branch:

  • compact package missing verifiers dependency. environments/compact/compact/harness.py imports verifiers.v1, but the package declared dependencies = []. Installing or publishing the harness wheel on its own would fail at import time. Now declares dependencies = ["verifiers"], matching the other v1 environment packages.

  • Legacy v0→v1 bridge dropped truncation. rollout_output_to_trace copied stop_condition from the v0 RolloutOutput but never accounted for v0's is_truncated flag. Trace.is_truncated is a derived property keyed off the v1 stop-condition vocabulary (max_turns, context_length, …) plus the final turn's finish_reason. v0 stop names (max_turns_reached, prompt_too_long, …) don't map onto that vocabulary, so legacy traces derived is_truncated=False even when the v0 rollout was truncated — undercounting truncation in eval/training metrics.

    Rather than store a flag on Trace (kept purely derived), the bridge now translates a truncated v0 rollout's stop name into v1's vocabulary so the property derives True:

    v0 stop condition v1 stop condition
    max_turns_reached max_turns
    prompt_too_long context_length
    timeout_reached harness_timeout
    max_total_completion_tokens_reached max_output_tokens
    (any other, when v0 is_truncated) max_output_tokens (fallback)

    An untruncated v0 rollout keeps its original stop_condition unchanged, so is_truncated still derives False (faithful to v0, including the v0 quirk where hitting max_turns with all-normal turns isn't flagged truncated).

Verification

Mapped a v0 rollout truncated via max-turns (v0 name max_turns_reached, not in v1's set) whose final turn ended normally (finish_reason="stop"):

trace.is_truncated
before False (wrong)
after True

Also checked: prompt_too_long/custom truncated stops derive True; the value survives the env-server wire path (model_dump()WireTrace.model_validate()); untruncated rollouts (incl. non-truncated max_turns_reached) still derive False; Trace gains no new field. tests/v1/test_trace.py passes.

Note for prime-rl

prime-rl's generation_truncated metric excludes the v0 name (stop_condition != "prompt_too_long"). With this change a prompt-overflow legacy truncation surfaces as context_length (matching native v1, which already emits context_length), so that exclusion should be updated to key off context_length to keep excluding prompt-overflow from generation truncation.


Note

Low Risk
Packaging fix plus localized legacy mapping logic; no new Trace fields or auth/data paths.

Overview
compact now lists verifiers in pyproject.toml so the harness can import verifiers.v1 when the wheel is installed on its own.

The v0→v1 rollout_output_to_trace path no longer copies stop_condition blindly. When v0 sets is_truncated, _v1_stop_condition rewrites known v0 stop names (max_turns_reached, prompt_too_long, etc.) into v1’s truncation vocabulary so Trace.is_truncated (derived from stop name + final finish_reason) is True. Untruncated rollouts keep the original stop string unchanged.

Reviewed by Cursor Bugbot for commit 73bae9e. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Fix v0 truncation stop condition mapping in legacy bridge and declare verifiers dependency for compact

  • In verifiers/v1/legacy.py, rollout_output_to_trace now maps v0 truncation stop names (e.g. max_turns_reached, prompt_too_long) to their v1 equivalents via a new _v1_stop_condition helper, falling back to max_output_tokens for unknown stops. Untruncated rollouts keep their original stop condition.
  • In environments/compact/pyproject.toml, adds verifiers as an explicit runtime dependency for the compact environment package.

Macroscope summarized 73bae9e.

mikasenghaas and others added 3 commits June 22, 2026 18:38
The compact harness package imports verifiers.v1 but declared
dependencies = [], so installing/publishing the wheel on its own could
fail at import time. Match the other v1 environment packages.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Trace.is_truncated is derived from the v1 stop-condition vocabulary and
the final turn's finish_reason. v0 stop names (e.g. max_turns_reached,
prompt_too_long) don't map onto that vocabulary, so legacy traces
reported is_truncated=False even when the v0 rollout was truncated.

Add an explicit, serialized `truncated` override on Trace (None for
native v1, so the derivation is unchanged) that is_truncated honors
first, and set it from the v0 rollout's own is_truncated flag in the
bridge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Keep Trace.is_truncated purely derived (no stored field). Instead, the
v0->v1 bridge translates a truncated v0 rollout's stop name into v1's
truncation vocabulary (max_turns_reached -> max_turns, prompt_too_long ->
context_length, ...; unmapped truncated stops fall back to
max_output_tokens) so the property derives True. Untruncated rollouts
keep their v0 stop condition unchanged.

This reverts the `truncated` field added to Trace in the previous commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mikasenghaas
mikasenghaas marked this pull request as ready for review June 22, 2026 18:49
@mikasenghaas
mikasenghaas merged commit 6e8b3cc into feat/nano-as-v1 Jun 22, 2026
4 checks passed
@macroscopeapp

macroscopeapp Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved

Straightforward bug fix ensuring truncation flags are preserved when bridging v0 rollout outputs to v1 Trace format. The change is limited to the legacy bridge with clear mapping logic, plus a simple dependency declaration.

You can customize Macroscope's approvability policy. Learn more.

pull Bot pushed a commit to Stars1233/verifiers that referenced this pull request Jun 23, 2026
…'s verifiers dep (PrimeIntellect-ai#1824)

* fix(v1): declare verifiers dep in compact harness package

The compact harness package imports verifiers.v1 but declared
dependencies = [], so installing/publishing the wheel on its own could
fail at import time. Match the other v1 environment packages.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(v1): carry v0 truncation flag through the legacy bridge

Trace.is_truncated is derived from the v1 stop-condition vocabulary and
the final turn's finish_reason. v0 stop names (e.g. max_turns_reached,
prompt_too_long) don't map onto that vocabulary, so legacy traces
reported is_truncated=False even when the v0 rollout was truncated.

Add an explicit, serialized `truncated` override on Trace (None for
native v1, so the derivation is unchanged) that is_truncated honors
first, and set it from the v0 rollout's own is_truncated flag in the
bridge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(v1): derive legacy truncation from stop condition, not a Trace field

Keep Trace.is_truncated purely derived (no stored field). Instead, the
v0->v1 bridge translates a truncated v0 rollout's stop name into v1's
truncation vocabulary (max_turns_reached -> max_turns, prompt_too_long ->
context_length, ...; unmapped truncated stops fall back to
max_output_tokens) so the property derives True. Untruncated rollouts
keep their v0 stop condition unchanged.

This reverts the `truncated` field added to Trace in the previous commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant