fix: carry v0 truncation flag through legacy bridge + declare compact's verifiers dep - #1824
Merged
mikasenghaas merged 3 commits intoJun 22, 2026
Merged
Conversation
The compact harness package imports verifiers.v1 but declared dependencies = [], so installing/publishing the wheel on its own could fail at import time. Match the other v1 environment packages. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Trace.is_truncated is derived from the v1 stop-condition vocabulary and the final turn's finish_reason. v0 stop names (e.g. max_turns_reached, prompt_too_long) don't map onto that vocabulary, so legacy traces reported is_truncated=False even when the v0 rollout was truncated. Add an explicit, serialized `truncated` override on Trace (None for native v1, so the derivation is unchanged) that is_truncated honors first, and set it from the v0 rollout's own is_truncated flag in the bridge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Keep Trace.is_truncated purely derived (no stored field). Instead, the v0->v1 bridge translates a truncated v0 rollout's stop name into v1's truncation vocabulary (max_turns_reached -> max_turns, prompt_too_long -> context_length, ...; unmapped truncated stops fall back to max_output_tokens) so the property derives True. Untruncated rollouts keep their v0 stop condition unchanged. This reverts the `truncated` field added to Trace in the previous commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mikasenghaas
marked this pull request as ready for review
June 22, 2026 18:49
Contributor
ApprovabilityVerdict: Approved Straightforward bug fix ensuring truncation flags are preserved when bridging v0 rollout outputs to v1 Trace format. The change is limited to the legacy bridge with clear mapping logic, plus a simple dependency declaration. You can customize Macroscope's approvability policy. Learn more. |
pull Bot
pushed a commit
to Stars1233/verifiers
that referenced
this pull request
Jun 23, 2026
…'s verifiers dep (PrimeIntellect-ai#1824) * fix(v1): declare verifiers dep in compact harness package The compact harness package imports verifiers.v1 but declared dependencies = [], so installing/publishing the wheel on its own could fail at import time. Match the other v1 environment packages. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(v1): carry v0 truncation flag through the legacy bridge Trace.is_truncated is derived from the v1 stop-condition vocabulary and the final turn's finish_reason. v0 stop names (e.g. max_turns_reached, prompt_too_long) don't map onto that vocabulary, so legacy traces reported is_truncated=False even when the v0 rollout was truncated. Add an explicit, serialized `truncated` override on Trace (None for native v1, so the derivation is unchanged) that is_truncated honors first, and set it from the v0 rollout's own is_truncated flag in the bridge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(v1): derive legacy truncation from stop condition, not a Trace field Keep Trace.is_truncated purely derived (no stored field). Instead, the v0->v1 bridge translates a truncated v0 rollout's stop name into v1's truncation vocabulary (max_turns_reached -> max_turns, prompt_too_long -> context_length, ...; unmapped truncated stops fall back to max_output_tokens) so the property derives True. Untruncated rollouts keep their v0 stop condition unchanged. This reverts the `truncated` field added to Trace in the previous commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two fixes to the v1 surface, both from review of the nano-as-v1 branch:
compactpackage missingverifiersdependency.environments/compact/compact/harness.pyimportsverifiers.v1, but the package declareddependencies = []. Installing or publishing the harness wheel on its own would fail at import time. Now declaresdependencies = ["verifiers"], matching the other v1 environment packages.Legacy v0→v1 bridge dropped truncation.
rollout_output_to_tracecopiedstop_conditionfrom the v0RolloutOutputbut never accounted for v0'sis_truncatedflag.Trace.is_truncatedis a derived property keyed off the v1 stop-condition vocabulary (max_turns,context_length, …) plus the final turn'sfinish_reason. v0 stop names (max_turns_reached,prompt_too_long, …) don't map onto that vocabulary, so legacy traces derivedis_truncated=Falseeven when the v0 rollout was truncated — undercounting truncation in eval/training metrics.Rather than store a flag on
Trace(kept purely derived), the bridge now translates a truncated v0 rollout's stop name into v1's vocabulary so the property derivesTrue:max_turns_reachedmax_turnsprompt_too_longcontext_lengthtimeout_reachedharness_timeoutmax_total_completion_tokens_reachedmax_output_tokensis_truncated)max_output_tokens(fallback)An untruncated v0 rollout keeps its original
stop_conditionunchanged, sois_truncatedstill derivesFalse(faithful to v0, including the v0 quirk where hittingmax_turnswith all-normal turns isn't flagged truncated).Verification
Mapped a v0 rollout truncated via max-turns (v0 name
max_turns_reached, not in v1's set) whose final turn ended normally (finish_reason="stop"):trace.is_truncatedFalse(wrong)TrueAlso checked:
prompt_too_long/custom truncated stops deriveTrue; the value survives the env-server wire path (model_dump()→WireTrace.model_validate()); untruncated rollouts (incl. non-truncatedmax_turns_reached) still deriveFalse;Tracegains no new field.tests/v1/test_trace.pypasses.Note for prime-rl
prime-rl's
generation_truncatedmetric excludes the v0 name (stop_condition != "prompt_too_long"). With this change a prompt-overflow legacy truncation surfaces ascontext_length(matching native v1, which already emitscontext_length), so that exclusion should be updated to key offcontext_lengthto keep excluding prompt-overflow from generation truncation.Note
Low Risk
Packaging fix plus localized legacy mapping logic; no new Trace fields or auth/data paths.
Overview
compactnow listsverifiersinpyproject.tomlso the harness can importverifiers.v1when the wheel is installed on its own.The v0→v1
rollout_output_to_tracepath no longer copiesstop_conditionblindly. When v0 setsis_truncated,_v1_stop_conditionrewrites known v0 stop names (max_turns_reached,prompt_too_long, etc.) into v1’s truncation vocabulary soTrace.is_truncated(derived from stop name + finalfinish_reason) isTrue. Untruncated rollouts keep the original stop string unchanged.Reviewed by Cursor Bugbot for commit 73bae9e. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Fix v0 truncation stop condition mapping in legacy bridge and declare
verifiersdependency for compactverifiers/v1/legacy.py,rollout_output_to_tracenow maps v0 truncation stop names (e.g.max_turns_reached,prompt_too_long) to their v1 equivalents via a new_v1_stop_conditionhelper, falling back tomax_output_tokensfor unknown stops. Untruncated rollouts keep their original stop condition.environments/compact/pyproject.toml, addsverifiersas an explicit runtime dependency for the compact environment package.Macroscope summarized 73bae9e.