feat(v1): episode-level derived aggregates and typed EnvInfo

[mikasenghaas]

Summary

• Episode gains derived, non-serialized aggregates mirroring Trace's property pattern: usage (agent-call usage summed via Usage.aggregate; judge/off-graph usage stays on trace.extra_usage), num_input_tokens / num_output_tokens / num_total_tokens, num_turns, and by_agent (traces grouped by agent name, e.g. n solvers). Consumers holding a validated WireEpisode (the serve wire, prime-rl) get them for free — properties stay off the wire and disk.

• Episode.env is now a typed EnvInfo carrying the env id, instead of a bare string.

• Episode.error is renamed to last_error, matching Trace.last_error; short docstrings added to all Episode fields and properties.

• StrictBaseModel is removed: the whole record tree (Trace, Episode, messages, usage, task data, timing, calls, nodes, judge/rubric types) parses non-strict — extras are ignored instead of rejected, so additive schema evolution no longer breaks older readers. WireTaskData keeps extra="allow" to round-trip unknown task fields.

• The dashboard's _tokens helper is deleted in favor of the Trace properties it re-implemented (num_input_tokens / num_output_tokens / num_branches / usage).

• uv.lock syncs the per-package exclude-newer cutoffs to the UTC timestamps pyproject pinned in chore: timezone-stable uv.lock and trace record cleanups #2172 (the lock was never re-locked, so every uv run --locked hook failed).

• The Timing.generation span is renamed Timing.agent (GenerationSpan → AgentSpan, still split into model and harness time), and Trace.split_generation becomes split_agent_time.

Breaking

• Timing.generation → Timing.agent (GenerationSpan → AgentSpan, Trace.split_generation → split_agent_time): persisted traces carry timing.agent; older records with timing.generation lose that span on re-read (extras are ignored).
• Episode.env is EnvInfo instead of str: episodes persist as "env": {"id": ...}; existing traces.jsonl files with string env no longer parse for resume/replay. Read episode.env.id for the id; Episode.of(trace, env=...) still takes the id string.
• Episode.error → Episode.last_error.
• vf.StrictBaseModel is gone (removed from the public API); subclass pydantic.BaseModel instead. No models reject unknown fields anymore: extras are ignored on validation (and dropped on rewrite paths like resume) rather than raising.

Verification

• uv run pytest tests/ green; ruff, pre-commit run --all-files, and the ty pre-push hook pass (re-run after the non-strict sweep).
• Live smoke: uv run eval reverse_text_v1 -n 2 --rich renders the usage/time rows and per-trace tokens correctly, pushes traces, and the persisted episode round-trips through read_episodes with env.id and working aggregates.

🤖 Generated with Claude Code

---

Note

High Risk
Persisted episodes and traces change shape (env object, timing.agent), and relaxing validation breaks strict readers and old traces.jsonl resume unless migrated; timing rename drops legacy generation spans on re-read.

Overview
Episode now stores env as EnvInfo ({"id": ...} on disk) instead of a plain string, exposes last_error (was error), and adds derived properties—usage, token/turn totals, and by_agent—summed from traces without extra wire fields.

Trace timing renames generation → agent (GenerationSpan → AgentSpan, split_generation → split_agent_time). Rollout, debug, legacy v0 mapping, and the eval dashboard follow the new span name and read token counts from Trace properties instead of a local _tokens helper.

StrictBaseModel is removed across traces, episodes, messages, tasks, judges, and artifacts; models use plain pydantic.BaseModel so unknown fields are ignored rather than rejected. Public exports drop StrictBaseModel and export AgentSpan instead of GenerationSpan.

Replay and e2e expectations use episode.env.id and the new serialized env shape.

^{Reviewed by Cursor Bugbot for commit c9ee978. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add episode-level derived aggregates and structured EnvInfo to Episode

• Episode now stores a structured EnvInfo object (with an id field) instead of a plain string for env, and exposes new aggregate properties: num_input_tokens, num_output_tokens, num_total_tokens, num_turns, usage, and by_agent.
• GenerationSpan is renamed to AgentSpan and Timing.generation becomes Timing.agent throughout traces, the dashboard, debug/replay CLI, and the v0-to-v1 legacy bridge.
• StrictBaseModel is removed across all v1 models; all Pydantic models now use BaseModel with extra='ignore' semantics, accepting unknown fields instead of raising validation errors.
• The error property on Episode is replaced by last_error; the dashboard and replay CLI are updated accordingly.
• Risk: serialized trace and episode JSON now uses agent instead of generation for timing fields and a nested env object instead of a string — existing consumers of these fields will break.

^{Macroscope summarized c9ee978.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR changes the serialization schema (Episode.env from string to typed object) and removes strict validation (extra="forbid") from all models, relaxing field constraints system-wide. These schema and validation changes warrant human review for backward compatibility implications.

^{You can customize Macroscope's approvability policy. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 515b820. Configure here.}