refactor(v1): reuse Pydantic adapters across validation - #2233
Conversation
ApprovabilityVerdict: Approved 04a2390 This refactoring consolidates duplicated Pydantic validation logic into shared functions and replaces manual checks with built-in Pydantic types (FiniteFloat, PositiveInt). The changes are mechanical DRY improvements with no new features or behavioral changes. Open review comments are minor naming style suggestions. You can customize Macroscope's approvability policy. Learn more. |
4d9cdd6 to
d31aa17
Compare
| from verifiers.v1.trace import Trace | ||
| from verifiers.v1.types import ID | ||
|
|
||
| _CriterionWeight = Annotated[float, Field(ge=0, allow_inf_nan=False)] |
There was a problem hiding this comment.
can we remove leading _ from al lthese
| return v | ||
|
|
||
|
|
||
| _CRITERIA_ADAPTER = TypeAdapter( |
| lambda value: ( | ||
| value.get("criteria", []) if isinstance(value, dict) else value | ||
| ) |
| ) | ||
|
|
||
|
|
||
| def _load_criteria( |
d31aa17 to
83efd59
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 83efd59. Configure here.
83efd59 to
010bee9
Compare
010bee9 to
b987707
Compare

Overview
Uses reusable Pydantic adapters where V1 handles schema-shaped data, while keeping shallow transport JSON parsing lightweight.
Details
Note
Low Risk
Mostly validation consolidation and adapter reuse; scoring semantics stay the same with stricter rejection of NaN/inf weights and malformed verdicts at config/parse time.
Overview
Pulls rubric criteria loading and verdict scoring into
load_criteriaandscore_verdictsinrubric.py, so the pluggedRubricJudgeand agentic-judge env share one path instead of duplicatedCriterionmodels and inline parsing infinalize().Config and rubric fields now reject bad numbers at parse time:
FiniteFloaton judge/score weights,CriterionWeight(non‑negative, finite) on rubric weights,PositiveIntfor RLMsummarize_at_tokens, andmax_criteriawithge=1. Agentic verdict JSON is validated throughRubricVerdictsbefore scoring.Performance/refactor:
functools.cachewrapsTypeAdapterfor CLI episode read/write; eachRolloutSessionexposes a cachedstate_adapterfor GET/PUT/state; legacy v0→v1 bridging validates tool defs via a sharedTOOLS_ADAPTERwithOnErrorOmitinstead of a manual loop.Reviewed by Cursor Bugbot for commit 04a2390. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Centralize rubric validation and reuse Pydantic
TypeAdapterinstances across v1load_criteriaandscore_verdictshelpers inrubric.pyto centralize criteria loading and verdict validation, replacing duplicated inline logic in the judge env andRubricJudge.CriterionWeight(non-negative, finite) and tightens field constraints onJudgeConfig,ScoreConfig, andRubricJudgeConfigto reject NaN/inf weights and non-positivemax_criteriaat parse time.TypeAdapterconstruction viacache(TypeAdapter)inoutput.pyand adds astate_adaptercached property toRolloutSessionso adapters are built once and reused.AgenticJudgeEnv.finalizeandRubricJudge.grade_batchnow strictly validate verdict names against the rubric and reject off-menu answers, where previously this may have passed silently.Changes since #2233 opened
TypeAdapterfactory and adapter instances from private naming convention to public naming convention [04a2390]Macroscope summarized b987707.