feat: store raw score and weight on trace rewards

[mikasenghaas]

Summary

• Trace.rewards values are now a Reward model carrying the raw score and its weight, instead of the pre-multiplied weight * score float — records keep raw scores readable and the weighted sum stays a derived view (Reward.value = score * weight, Trace.reward = sum of values).
• record_reward stores the pair; TRACE_VERSION bumped 3 → 4; Reward exported from verifiers.v1.
• Platform uploads flatten sub-reward keys and aggregate per-name means over the raw score (matching v0's per-function outputs); the headline reward / avg_reward remain the weighted sum.
• The agentic-judge env's task_weight rescale now scales each reward's weight in place of mutating the product, so the solver's raw task scores stay visible on the trace.
• Dashboard breakdown shows raw-score means; legacy bridge and the proposer-solver taskset read the new shape; e2e/judge test assertions updated.

Breaking

• Trace record schema: rewards changes from dict[str, float] to dict[str, {score, weight}]; TRACE_VERSION 3 → 4. Old v3 records no longer parse through Trace/WireTrace, so --resume/replay of runs written before this change fails on the rewards field (no coercion shim by design). Migration: re-run the eval, or rewrite old records' reward values as {"score": <value>, "weight": 1.0}.
• Consumers reading trace.rewards values as floats must switch to .score (raw) or .value (weighted contribution).

Verification

Ran uv run eval reverse_text_v1 --no-push before and after:

• Before: rewards: {"lcs": 1.0} (version 3)
• After: rewards: {"lcs": {"score": 1.0, "weight": 1.0}} (version 4), headline reward=1.000 unchanged; WireTrace.model_validate round-trips the new record.

pytest tests/v1 -m "not e2e", ruff, and pre-commit all pass.

🤖 Generated with Claude Code

---

Note

Medium Risk
Breaking trace schema and any code treating trace.rewards values as floats; headline reward/avg_reward behavior is preserved but resume/replay of v3 records fails without migration.

Overview
Trace rewards now persist as Reward objects (score + weight) instead of pre-weighted floats. Trace.reward is derived as the sum of score * weight; record_reward writes the pair unchanged. TRACE_VERSION bumps 3 → 4 — serialized traces with the old dict[str, float] shape no longer validate without migration.

Downstream readers are updated to use .score (or .weight where composition matters): eval dashboard reward breakdowns, platform push flattening and per-name means, legacy v0 bridge, proposer-solver solve rate, and agentic-judge task_weight (scales each reward’s weight in place instead of multiplying stored values). Tests assert the new shape, including separate raw score vs weight when env score weights apply.

^{Reviewed by Cursor Bugbot for commit 18ea5a1. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Store raw score and weight on trace rewards as Reward objects

• Introduces a Reward model in verifiers/v1/trace.py with score, weight, and a computed value (score * weight) field, replacing the previous dict[str, float] type for Trace.rewards.
• Bumps TRACE_VERSION from 3 to 4 to reflect the schema change.
• Updates all call sites (push.py, legacy.py, agentic_judge/env.py, dashboard eval.py, and proposer_solver_v1) to access .score or .weight rather than treating reward values as raw floats.
• Exports Reward from the verifiers.v1 package.
• Behavioral Change: Any code reading trace.rewards[name] as a float will break; callers must now use .score, .weight, or .value.

^{Macroscope summarized 18ea5a1.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR introduces a new Reward data model and changes the trace schema (version 3→4), replacing dict[str, float] with dict[str, Reward]. Breaking schema changes and new data models warrant human review to validate migration impact.

^{You can customize Macroscope's approvability policy. Learn more.}

[hallerite]

lol since when do we have this

[mikasenghaas]

i will reset this to 1 and only use it once we have trace api lol

[mikasenghaas]

but eventually it will be important and we have to think abt versioning