P2-13: Stage-1 temperature scaling + the non-gated reliability read (T = 0.9896)

docs/decisions/ADR-0005-non-circular-eval-design.md

@@ -581,3 +581,42 @@ Rationale for provisional-over-reorder: reordering (pinning the final D3 operati
 **Scope guard (§10.3):** records the measured envelope only; pins no threshold / operator / power-floor / sparing change; authorizes no submit; changes no code; flips no flag.
 **§10.1 evidence gate:** N/A — compute-budget arithmetic on measured wall (ordinary engineering); asserts no biological/statistical fact.
 **Sign-off (A10 Phase-2):** ☑ recorded 2026-07-28 (bioedca), CLAUDE.md §7 item 2 — per-round producer budget pinned on the measured round-0 numbers (N₀ = 941; ≈ 4.42 CPU-core-h/round; array width 48 × cpus-per-task 2; ~7 min wall/round; 0 GPU-h). The N = 4 RUN is deferred (structural-0-yield; `MSA_SUPPLY_AVAILABLE` stays False); envelope documented, not exercised.
+
+## Amendment A11 — Stage-1 temperature scaling: one shared scalar T on the per-nucleotide 8-class logits, fitted by an exact convex 1-D solve, applied **before** the D3/A3 reconciliation operator; the P2-13 reliability read is per-class one-vs-rest, **non-gated**, and fitted/read on cluster-disjoint halves of `selection_val` because the D11 calibration split does not exist until P3-02 (P2-13, 2026-07-30)
+
+- **Status:** **Accepted (user sign-off 2026-07-30; CLAUDE.md §7 item 2, bioedca — "sign A11 and land the whole step now", with the fold and T-form options taken as drafted).** Extends **D11** to Stage 1. Pins no new numeric gate default and moves no gate.
+- **Label note:** `A11`. `A9` = P2-10e, `A10` = P2-10e-msa-producer. The pre-existing duplicate-`A3` collision (P0-28 and P2-03) is untouched and still owed at the P2 exit gate.
+
+**Why.** D11 pins the recalibration stack order (*train → temperature-scale on a disjoint calibration split → prior-shift*), the binned-ECE estimator (**15 equal-mass bins, debiased**), and the gated object (the *named posterior*, pre-prior-shift, graded at **P3 exit**) — but it pins them **"on the positive-class posterior"**, i.e. for Stage-2's binary head (`ADR-0005:111`; ADR-0001 D1 gives Stage 2 the *"calibrated binary"* head). Stage 1 emits **per-nucleotide 8-class logits** (`ADR-0001:31`). Four things were therefore genuinely open for P2-13, each verified absent from `PRD.md` and all six ADRs:
+
+1. the **calibration axis** for a per-nt multi-class segmenter (`PRD.md` has no sentence on it; `isotonic`, `Platt`, `reliability diagram` = 0 hits in both PRD and ADRs);
+2. the **parameterisation** — one shared scalar T vs per-class (vector) T (`single-T`, `vector scaling` = 0 hits);
+3. the **fit objective/optimiser** (`LBFGS` = 0 hits; `imp.md`'s *Notes* suggest PyTorch LBFGS — a suggestion, not a pin);
+4. **which fold** the P2-13 fit uses — D11's *"disjoint calibration split"* is the **only** occurrence of that phrase in the PRD (`PRD.md:248`), ADR-0004's fold vocabulary contains **no** calibration fold, the committed split table has **no `calib` column**, and `imp.md` assigns that carve to **P3-02** (*"a disjoint calibration subset carved from the training folds, whole-cluster-assigned"*).
+
+**Pin 1 — axis: the per-nucleotide 8-class logit vector; the read reduces one-vs-rest per class.** T is fitted on the same axis the loss trains (`PRD.md:236`, per-nucleotide cross-entropy). The reliability read calls the **frozen** `metrics.binned_ece` (15 equal-mass, debiased) **once per class** on the one-vs-rest posterior, and reports the **per-class table** plus the **worst (max) over the three ADR-0004 D6 core elements** {Stem I, Specifier, Antiterminator}. **No cross-class mean** is reported — mirroring GATE-4's *"no cross-unit mean"* discipline (`PRD.md:246`); the only macro-average the PRD pins for ECE is **over held-out orders** (`PRD.md:247`), a P3/P4 statistic, not this one. `binned_ece` is **called, never forked**; P2-13 promotes the bin table out of it as `metrics.reliability_bins` so the plotted curve and the reported number are one arithmetic, and the committed hand-computed expectations in `tests/ml/test_eval_gate.py` (reproduced **bit-exactly** across that refactor) remain the non-tautological guard.
+
+**Pin 2 — parameterisation: ONE shared scalar T > 0, applied as `logits / T`.** The method of the paper D11 already cites (arXiv:1706.04599), whose vector/matrix-scaling variants underperform it. Per-class (vector) scaling is **not** adopted; adopting it later is its own amendment, because it is the variant that can move an arg-max at will and would therefore re-open GATE-4's graded quantity.
+
+**Pin 3 — fit: an exact, deterministic 1-D convex solve in β = 1/T; not SGD, not LBFGS.** With `β = 1/T`, `NLL(β) = Σ_i [ logsumexp_c(β·z_ic) − β·z_{i,y_i} ]` is **convex in β** (log-sum-exp composed with a linear map, less a linear term) and `dNLL/dβ = Σ_i [ Σ_c p_ic·z_ic − z_{i,y_i} ]` is monotone increasing, so the fit is a **bracketed bisection on a monotone derivative** — exactly reproducible, **no seed, no learning rate, no optimiser state**, and importable in the bare-CI tier (numpy only), so the unit gate is **not** torch-gated (CI installs no torch). **Fail-closed, in both degenerate directions, each guarded separately:** the exact `β → ∞` limit `Σ_i (max_c z_ic − z_{i,y_i})` is 0 **iff** every position is already its own arg-max (no minimiser exists; T → 0) — checked exactly because the *sampled* gradient underflows to 0.0 there and a `|grad| ≤ tol` rule would read that flat tail as stationarity and certify an arbitrary T; and a sign check at each bracket endpoint refuses a minimiser lying outside `T ∈ [1e-3, 1e3]`. Neither returns an endpoint dressed as a solution.
+
+**Pin 4 — where T sits relative to the frozen reconciliation operator: BEFORE it, and the arg-max consequence is MEASURED, not assumed.** The D3+A3 operator consumes **per-window logits** and internally takes `log_softmax` per window before the coverage-normalised average (`ADR-0005:284`), so the calibrated deployment order is `logits / T → reconcile → posterior`. Two consequences, one counter-intuitive:
+
+- a single T **preserves each window's own arg-max exactly** (strictly monotone);
+- it **does not necessarily preserve the *reconciled* arg-max** where coverage ≥ 2, because the operator averages *probabilities* across overlapping windows and re-tempering the summands can change which class wins that average.
+
+So temperature scaling is **not** a priori GATE-4-neutral, and the P2-12 flag (*"check whether temperature scaling stabilises the Antiterminator min-core-F1 under the frozen D3 arg-max operator"*) is an **empirical** question, not one answerable by invariance. The P2-13 artifact therefore reports `n_positions_argmax_changed`, `n_positions_coverage_ge_2`, and the per-class F1 / boundary-IoU delta. **GATE-4 at P2-14 is graded on the uncalibrated posterior** unless a later signed decision says otherwise; A11 changes no gated quantity.
+
+**Pin 5 — the P2 fold, and what the P2-13 numbers may and may not be called.** No D11-disjoint calibration split exists at P2. Rather than invent one or silently reuse a selection fold:
+
+- **Fit and read on cluster-disjoint halves of the P2-06a `selection_val` inner rung** (830 records / 469 clusters): a **whole-cluster** seeded halving (`CALIB_SPLIT_SEED = 20260730`, `fraction = 0.5`, drawn to a record-count target exactly as `selection_val_cluster_ids` draws its rung) — T fitted on half A, reliability read on half B — so the read is **out-of-sample for T**. Because two ~equal halves make a swapped fit/read wiring invisible to every count in the artifact while turning the read in-sample for T, the halves are validated by **identity**: the report carries the cluster→size mapping and its validator **re-cuts** the fit half from it, rather than reading back a boolean.
+- **Mandatory disclosures, shipped as report constants (not prose):** (i) `selection_val` is the fold the P2-06 sweep **selected on**, so it is **NOT** the D11 disjoint calibration split; (ii) **no T from P2-13 is shipped or consumed by any gate** — GATE-2's T is re-fitted at **P3-02/P3** on the `calib` carve; (iii) the report carries `gated: false`, `is_science: false`, and every ECE with a **cluster-blocked** bootstrap CI; (iv) the CI is a **200-replicate** percentile interval (the frozen stdlib estimator costs ~0.6 s per call at this N, so the D5 default of 2000 would be hours of pure-Python sorting for a non-gated read) over ~235 clusters, and is disclosed as coarse; (v) the fit objective is the **per-window** NLL while the read is on the **reconciled** posterior — a deliberate mismatch (convexity on one side, the deployment object on the other), with both reported; (vi) the per-nt class mix is background-dominated, so the pooled-NLL fit is background-dominated.
+- **The prior-shift stage is refused fail-closed at P2.** The artifact stops at the *named posterior*; the Saerens/Elkan correction is P5. The stack-order record P2-13 owes is a machine-checked constant + validator clause, not a sentence.
+
+**Alternatives rejected.** (a) **Pull P3-02's `calib` carve forward** — re-opens the committed, git-LFS split table and its closed schema and forces a full-corpus no-leakage re-run (§8.5) to serve a **non-gated** read, duplicating a step `imp.md` assigns elsewhere. (b) **Fit and read on all of `selection_val`** — the read becomes in-sample for T and flatters the machinery it exists to check. (c) **Machinery-only on synthetic fixtures** — `imp.md`'s own P2-13 gate asks for a reliability read on real Stage-1 per-class posteriors, and the measured local cost is ≈46 s of laptop GPU, so withholding it would be a non-delivery without a cost justification.
+
+**Not amended.** D11's gated object, threshold (**ECE ≤ 0.05**), estimator (15 equal-mass debiased bins) and **P3-exit** grading point; D11's stack order; D13/A2's distinct small-N-robust OOD estimator and `OOD_ECE_MIN_N = 20`; D3/A3's frozen reconciliation operator (**consumed**, not modified); D6/GATE-4's definition and its P2-14 grading on the uncalibrated posterior; ADR-0004's fold vocabulary and the committed split-table schema (**untouched** — A11 adds no column).
+
+**§10.1 evidence gate.** The one methodological claim carrying a citation is that post-hoc **single-parameter temperature scaling** is the appropriate multi-class recalibration form here — arXiv:1706.04599 (Guo et al. 2017), already cited by D11 and by `power.CITATIONS["calibration_modern"]`, which introduces it and reports vector/matrix scaling as weaker on the same benchmarks; the binned-estimator lineage is PMID:25927013 (`power.CITATIONS["calibration_binning"]`). The convexity of `NLL(β)` and the strict monotonicity of `logits/T` are mathematical facts derived in Pin 3/Pin 4, not empirical claims, and the arg-max behaviour **through** the averaging operator is left as a measurement rather than asserted.
+
+**Cross-reference impact.** **P2-13** (this step — `src/tbox_finder/calib/temperature.py`, `metrics.reliability_bins`, `reports/p2/calibration.json`, `figures/calib/`, `tests/unit/test_temperature.py`); **`imp.md` P3-02** — must **re-fit** T on the `calib` carve; A11's T is a machinery demonstration and is **not** inherited; **P3 exit / GATE-2** — grades D11 unchanged, on the P3-02 split, with this estimator; **P2-14 / GATE-4** — graded on the uncalibrated posterior, with `n_positions_argmax_changed` making any future decision to grade the calibrated one an explicit choice rather than a drift; **P2-12** — its flagged Antiterminator IoU fragility is addressed here **as a measurement**, and is not closed by single-T invariance (Pin 4); **ADR-0001 D1** — its *"P2-trained, graded GATE-4/GATE-2"* wording (`ADR-0001:35`) reads as if a calibration gate lands at P2, while D11 (`ADR-0005:105/:109`) and `PRD.md:56/:358` put the ECE gate at **P3 exit**; A11 follows the PRD/D11 reading and flags the ADR-0001 phrasing for reconciliation at the P2 exit gate alongside the duplicate-`A3` cleanup; **P2 exit gate** — `README.md` + `paper/manuscript.qmd` owe this step a line, batched there per §6.4.