[GDPVal] Harden production rollout and judging paths

[Kh4L]

Summary

• Route malformed, truncated, or scoreless judge responses out of headline metrics and through retry-aware failure sidecars; preserve completed policy artifacts for re-judging and make deterministic payload/context failures terminal.
• Bound GDPVal text, binary, Office, PDF, media, and ZIP handling before reads and request construction; preserve sidecar provenance, attachment fairness, and A/B swap invariance while avoiding repeated hashing for payloads that already fit.
• Make standard and multi-stage collection honor finite dispatch budgets, stage-aware longest-first scheduling, attempt accounting, and cached-deliverable reuse; harden resume with config fingerprints, legacy migration, completion coverage checks, atomic writes, and stale-metric cleanup.
• Allow opt-in bounded partial completion for non-final calibration stages after persisted timeouts, with overall and per-reference evidence floors, balanced fresh assignments, frozen evidence snapshots, and fail-closed resume. The final stage still requires full evidence for a headline ELO.
• Sanitize unparsed raw tool-call blocks before provider requests and guarantee globally unique converted tool-call IDs without mutating stored trajectories.
• Align reverification with the new judge-invalid failure contract, preserve file/reference context, migrate legacy invalid rows, and reject unsafe multi-stage judge-failure recovery.

Behavior changes

• dispatch_budget_s now requires a finite positive num_samples_in_parallel.
• Missing rubrics and deterministic request-size/context failures are terminal instead of consuming repeated judge attempts.
• --judge-failed-only fails fast for multi-stage sidecars; multi-stage collection resume is the supported recovery path for those rows.
• partial_completion is opt-in and valid only for non-final stages. Resume uses the same output path plus --resume; changing a policy after a partial outcome has been frozen fails closed.

Validation

• env PYTHONPATH=. uv run pytest resources_servers/gdpval/tests -q
  • 310 passed, 9 optional-dependency skips
• env PYTHONPATH=. uv run pytest tests/unit_tests/test_deliverables.py tests/unit_tests/test_dispatch_latency.py tests/unit_tests/test_rollout_collection.py tests/unit_tests/test_rollout_reverification.py -q
  • 229 passed
• Focused multi-stage suite, including 41/45 calibration recovery, interrupted-final-stage resume, stale downstream invalidation, frozen-evidence validation, and crash-order handling:
  • 135 passed
• Ruff check and format check pass for every changed Python file.
• git diff --check passes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[agronskiy]

@Kh4L it seems for me, the logic prevents timing out-tasks (the ones that could not finish in 3.5 hours cut-off) of stage-1 prevent starting stage-2. I think we should admit that some GDPVal tasks will timeout on stage-1 as either hard or smth like that -- and this should not prevent apporximately calibrating and starting stage-2. WDYT?

[Kh4L]

Implemented the Stage 1 timeout recovery in 14125b765.

• A non-final stage can now opt into bounded partial completion with overall and per-reference coverage floors.
• An existing 41/45 Stage 1 run with four persisted timeouts can advance on --resume without redispatching those four long-running tasks.
• Fresh policy-enabled stages balance task assignments across the selected references.
• The accepted evidence and omitted keys are frozen, so later resume cannot silently change Stage 2 reference selection.
• The final stage remains strict: an incomplete final stage emits no headline ELO and resume dispatches only its missing rows.

The recovery configuration and --resume/RERUN_INCOMPLETE distinction are documented in benchmarks/gdpval/README.md. The exact 41/45 → partial calibration → interrupted final stage → single-row final resume path is covered by the focused suite.