fix(routing): fail-closed ablation θ and persist streamed snapshots

[cursor]

Buyer next action

Prefer this PR over #650 and #631. Run python tests/test_reasoning_effort_profile.py. Keep live route/conduct defaults unchanged while production_default_change_allowed is false. Do not raise temperature to stand in for reasoning effort.

Why this PR

#650 made θ̂ = (1−λ)θ and RMSE(θ̂, θ) honest, but three fail-closed / persist seams were still open:

• run_equal_budget_ablation used float() and laundered boolean or string θ into an RMSE report labeled usage_source=synthetic_true_theta.
• The Hypothesis/Atheris target put true_theta inside the profile payload, so parse failed before the ablation asserts ever ran.
• stream_route attached the snapshot in memory only. With --state-db, a restart dropped the streamed run while run and batch_route survived.

What landed

• Ablation true-θ uses the same _reject_non_finite_number helper as estimate_theta.
• The fuzz target pops true_theta before parse and always exercises the ablation after a valid profile.
• stream_route writes the run (and its reasoning_effort_snapshot) through --state-db.
• Tests cover boolean/string θ rejection and streamed snapshot reload after restart.

Doctoring (APA 7th)

• Sakana AI. (2026). Sakana Fugu Technical Report.
• Xu, J., Sun, Q., Schwendeman, P., Nielsen, S., Cetin, E., & Tang, Y. (2025). Trinity: An evolved LLM coordinator (arXiv:2512.04695).
• Nielsen, S., Cetin, E., Schwendeman, P., Sun, Q., Xu, J., & Tang, Y. (2025). Learning to orchestrate agents in natural language with the Conductor (arXiv:2512.04388).
• Baker, F. B. (2001). The basics of item response theory (2nd ed.). https://eric.ed.gov/?id=ED458219

Out of scope (do not fold in)

• Tools-path honesty, SSE proxy, KV host allowlist, or gateway Bearer-token KV (feat(api): SSE-proxy tools and response_format on chat completions #606–fix(api): fail-closed whitespace-only mode on tools passthrough #629, fix(api): pop null json_schema description/strict omit-real before proxy #646–fix(api): accept official Responses text.format omit-real #657).
• Changing OrchestrationPolicy defaults.
• Live NVIDIA NIM calls (issue [Product Gap] Evidence-grade NVIDIA NIM model discovery and cost-quality benchmark #86).
• Applying the catalog to ModelClient request knobs (next buyer gap after this lands).

Verify

python tests/test_reasoning_effort_profile.py
python tests/test_paper_contracts.py
python tests/test_conventions.py
python tests/test_persistence.py

  

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

[opencode-agent]

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

• Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

• Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

• Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

• Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

• Result: REQUEST_CHANGES

• Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head b9363042ad8a9be2bd4faca2449365fc44d65be7.

• Head SHA: b9363042ad8a9be2bd4faca2449365fc44d65be7

• Workflow run: 32124537618

• Workflow attempt: 1

Coverage evidence

Coverage evidence job did not run or did not publish coverage evidence.

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: fuzz.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: fuzz.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file (10 files)"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file (10 files)"]
  R2 --> V2["required checks"]
  Evidence --> S3["Docs (5 files)"]
  S3 --> I3["operator or user guidance"]
  I3 --> R3["Review risk: Docs (5 files)"]
  R3 --> V3["docs review"]
  Evidence --> S4["Test (2 files)"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test (2 files)"]
  R4 --> V4["targeted test run"]

Loading

[opencode-agent]

OpenCode Review Overview

• Head SHA: b9363042ad8a9be2bd4faca2449365fc44d65be7
• Workflow run: 32129300630
• Workflow attempt: 1
• Gate result: REQUEST_CHANGES (approval step)

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

• Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

• Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

• Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

• Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

• Result: REQUEST_CHANGES

• Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head b9363042ad8a9be2bd4faca2449365fc44d65be7.

• Head SHA: b9363042ad8a9be2bd4faca2449365fc44d65be7

• Workflow run: 32129300630

• Workflow attempt: 1

Coverage evidence

Coverage evidence job did not run or did not publish coverage evidence.

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: fuzz.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: fuzz.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file (10 files)"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file (10 files)"]
  R2 --> V2["required checks"]
  Evidence --> S3["Docs (5 files)"]
  S3 --> I3["operator or user guidance"]
  I3 --> R3["Review risk: Docs (5 files)"]
  R3 --> V3["docs review"]
  Evidence --> S4["Test (2 files)"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test (2 files)"]
  R4 --> V4["targeted test run"]

Loading

[opencode-agent]

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

• Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

• Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

• Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

• Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

• Result: REQUEST_CHANGES

• Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head b9363042ad8a9be2bd4faca2449365fc44d65be7.

• Head SHA: b9363042ad8a9be2bd4faca2449365fc44d65be7

• Workflow run: 32129300630

• Workflow attempt: 1

Coverage evidence

Coverage evidence job did not run or did not publish coverage evidence.

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: fuzz.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: fuzz.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file (10 files)"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file (10 files)"]
  R2 --> V2["required checks"]
  Evidence --> S3["Docs (5 files)"]
  S3 --> I3["operator or user guidance"]
  I3 --> R3["Review risk: Docs (5 files)"]
  R3 --> V3["docs review"]
  Evidence --> S4["Test (2 files)"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test (2 files)"]
  R4 --> V4["targeted test run"]

Loading