diff --git a/docs/feature-posterior-temperature.md b/docs/feature-posterior-temperature.md new file mode 100644 index 000000000..adc1adf29 --- /dev/null +++ b/docs/feature-posterior-temperature.md @@ -0,0 +1,132 @@ +# Feature spec: γ rerank — Boltzmann posterior temperature (#796) + +**Status:** implemented behind default-OFF flag at v3.x; bench panel widened (`ordered_top_k_overlap`, `rank_biased_overlap`); adoption verdict (flip default) **deferred** until a labeled relevance corpus exists. +**Issue:** #796 +**Substrate prereqs:** #756 / #757 / #759 / #760 (meta-belief consumer pattern, shipped v3.x). Operator path-B decision 2026-05-14T16:48Z, refined to γ″ after the R&D campaign 2026-05-14T18:04Z. + +--- + +## Purpose + +The v1.3 retrieval rerank is **log-additive**: + +``` +score = log(max(-bm25_raw, EPS)) + posterior_weight · log(posterior_mean) +``` + +`posterior_weight` is a process-wide scalar tuned at v1.3.0 (default `0.5`). Two independent operator concerns motivate the γ surface: + +1. The intended consumer for `meta:retrieval.posterior_temperature` (a softmax / Boltzmann temperature `T`) does not exist on `github/main` — pre-claim analysis on #758 confirmed `git grep -E 'softmax|boltzmann|temperature' github/main -- 'src/aelfrice/'` returns zero hits in src. Issue #796 is the load-bearing precursor: ship the temperature surface so `meta:retrieval.posterior_temperature` has something to bind to. +2. PR@5 + Spearman ρ alone cannot discriminate top-K reorderings from middle-of-list churn (#796 R4). The bench panel needs `ordered_top_k_overlap` and `rank_biased_overlap` to make γ-vs-log-additive comparisons load-bearing. + +γ is **reparametrised log-additive**, not a new family — see *Contract* below. At `T = 1.0` it is byte-identical to `partial_bayesian_score(..., posterior_weight=1.0)`. Lower `T` sharpens the posterior contribution; higher `T` flattens toward BM25-only ranking. + +--- + +## Contract + +```python +from aelfrice.scoring import gamma_posterior_score + +score: float = gamma_posterior_score( + bm25_raw, alpha, beta, temperature, +) +``` + +Formula: + +``` +score = log(max(-bm25_raw, EPS)) + (1 / T) · log(posterior_mean(α, β)) +``` + +Equivalent to `partial_bayesian_score(bm25_raw, α, β, posterior_weight=1.0/T)` exactly — γ is a reparametrisation, not a new function family. Pure, deterministic, no store reads, no clock reads. + +`T <= GAMMA_TEMPERATURE_FLOOR` (1e-6) clamps upward so a misconfigured meta-belief or env override never raises at retrieval time. Negative temperatures are likewise clamped (the Boltzmann reading is undefined for `T <= 0`). + +--- + +## Flag + meta-belief + +Resolved at `retrieve()` / `retrieve_with_tiers()` entry, once per call. + +| Layer | Surface | Resolver | +|---|---|---| +| Env | `AELFRICE_USE_GAMMA_POSTERIOR_TEMPERATURE` | `_env_use_gamma_posterior_temperature_override()` | +| TOML | `[retrieval] use_gamma_posterior_temperature` | `_read_toml_flag_for(...)` | +| Default | False | `resolve_use_gamma_posterior_temperature()` | + +When the flag resolves True, the temperature is resolved against the meta-belief substrate: + +```python +T = resolve_posterior_temperature_with_meta(store, now_ts=...) +``` + +Bounds: `T ∈ [POSTERIOR_TEMPERATURE_FLOOR, POSTERIOR_TEMPERATURE_CEIL] = [0.5, 2.0]`. Log-linear decode from the meta-belief's `[0, 1]` posterior value; geometric mean is exactly 1.0, so the cold-start `static_default = 0.5` decodes to `T = 1.0` and a fresh install with the flag on is byte-identical to `partial_bayesian_score(..., 1.0)`. Adaptive learning of `T` (the evidence-signal loop that moves the meta-belief away from its prior) is out of scope for #796 — that is issue #758. + +When the flag is False, `gamma_temperature` is `None` and `_l1_hits` skips the γ branch entirely. The pre-#796 log-additive contract holds byte-for-byte. + +### Heat-rerank composition + +γ and the heat-kernel rerank (`use_heat_kernel`) are **mutually exclusive** on a given call. When both flags are on and a non-stale eigenbasis is available, the heat-rerank fires and γ is a no-op for that call. Composition was deferred to a later issue by the operator decision (R&D campaign verdict, 2026-05-14T18:04Z). + +--- + +## Where γ sits + +`src/aelfrice/retrieval.py::_l1_hits` — the rerank loop, both branches: + +```python +elif gamma_temperature is not None: + s = gamma_posterior_score( + bm25_raw, b.alpha, b.beta, gamma_temperature, + ) +else: + s = partial_bayesian_score( + bm25_raw, b.alpha, b.beta, posterior_weight, + ) +s = _hash_n_boosted(s, b.content, hash_n_literals) +``` + +`_hash_n_boost` runs after the rerank, unchanged from the log-additive path — the R2 / R2b finding that path-B-literal divergence was entirely the boost interaction is informational, not a code change. + +The byte-identical short-circuit (`posterior_weight == 0.0 and not heat_active and not hash_n_literals`) extends to require `gamma_temperature is None` so γ-on always exercises the rerank loop. + +--- + +## Bench-gate / ship-or-defer policy + +| Gate | Status | Notes | +|---|---|---| +| **G1** — surface lands behind a default-OFF flag | shipped | this PR | +| **G2** — bench panel widened with `ordered_top_k_overlap` + `rank_biased_overlap` | shipped | `src/aelfrice/calibration_metrics.py`, `src/aelfrice/eval_harness.py::compare_ranking_panel` | +| **G3** — labeled relevance corpus exists | **pending** | corpus authoring tracked separately | +| **G4** — γ@T=1.0 vs γ@T=0.5 / T=2.0 on labeled corpus shows discriminable rank-overlap deltas | **pending G3** | adoption verdict gate | +| **G5** — flip default to True if G4 clears with effect size ≥ 1σ | **pending G3, G4** | follow-up PR | + +The bench-gate / ship-or-defer policy is the same shape as `feature-type-aware-compression.md` § *"Bench-gate / ship-or-defer policy"* and `feature-bfs-multihop.md` § *"Bench-gate posture"*. Until G3 lands, the flag is off and γ is plumbing — no behavioural change on any default code path. + +--- + +## Out of scope (separate issues) + +- **Adaptive `T`** — the evidence-signal loop that moves the meta-belief away from its `static_default = 0.5` prior. That is #758. Until #758 ships, the meta-belief is never updated; flag-on cold installs decode to `T = 1.0` and stay there. +- **ζ follow-up** — a bounded / sigmoid posterior-contribution parametrisation that gives `T` subtler dynamics than γ's global re-weighting. Filed as a separate R&D campaign after R&D refuted the α/β/γ trichotomy (see #800). +- **Composition with heat-rerank** — both flags can be on but γ is a no-op on heat-active calls. Composing the two scoring paths is a separate scoping decision. +- **Composition with `_hash_n_boost`** — boost interaction is informational, not a code change. R2 / R2b finding. + +--- + +## Refs + +- #796 — this issue (operator path-B decision 2026-05-14T16:48Z; γ″ refinement 2026-05-14T18:04Z). +- #758 — adaptive `T` follow-up; gated on #796 shipping. +- #800 — ζ parametrisation R&D campaign. +- #605 — PHILOSOPHY (deterministic, narrow surface). γ inherits. +- #661 — federation read-only; meta-belief is local-only write state. +- `src/aelfrice/scoring.py:gamma_posterior_score` — entry point. +- `src/aelfrice/retrieval.py:resolve_use_gamma_posterior_temperature` — flag resolver. +- `src/aelfrice/retrieval.py:resolve_posterior_temperature_with_meta` — decoder. +- `src/aelfrice/retrieval.py:_l1_hits` — call site. +- `src/aelfrice/calibration_metrics.py:ordered_top_k_overlap`, `rank_biased_overlap` — bench primitives. +- `src/aelfrice/eval_harness.py:compare_ranking_panel` — bench-panel aggregator. +- `tests/test_scoring_gamma.py`, `tests/test_retrieve_gamma_flag.py`, `tests/test_rank_overlap_metrics.py` — contract tests. diff --git a/src/aelfrice/calibration_metrics.py b/src/aelfrice/calibration_metrics.py index ac2717e93..13de98ec7 100644 --- a/src/aelfrice/calibration_metrics.py +++ b/src/aelfrice/calibration_metrics.py @@ -27,6 +27,8 @@ "precision_at_k", "roc_auc", "spearman_rho", + "ordered_top_k_overlap", + "rank_biased_overlap", ) @@ -130,3 +132,97 @@ def spearman_rho( if denom == 0: return None return (n * sxy - sx * sy) / denom + + +def ordered_top_k_overlap( + a: Sequence[object], b: Sequence[object], k: int, +) -> float: + """Fraction of the top-k positions where ``a`` and ``b`` agree. + + Compares the first ``k`` elements of two ranked lists position-by- + position; the score is the count of matching positions divided by + ``k``. Returns 1.0 when both lists' prefixes are identical and 0.0 + when no top-k position matches. + + Items beyond rank ``k`` are ignored. Missing slots (an input shorter + than ``k``) count as non-matches. Use this alongside + `rank_biased_overlap` to discriminate top-of-list churn from + middle-of-list reorderings — the #796 R4 finding was that PR@k and + Spearman ρ alone cannot tell those apart. + """ + if k <= 0: + raise ValueError("k must be positive") + a_top = list(a[:k]) + b_top = list(b[:k]) + matches = sum( + 1 + for i in range(k) + if i < len(a_top) and i < len(b_top) and a_top[i] == b_top[i] + ) + return matches / k + + +def rank_biased_overlap( + a: Sequence[object], b: Sequence[object], p: float = 0.9, +) -> float: + """Rank-biased overlap (RBO) — top-weighted ranking similarity. + + Implements the extrapolated finite-list form (RBO_EXT) from Webber + et al. (2010), "A Similarity Measure for Indefinite Rankings". + Comparison runs up to depth ``D = min(len(a), len(b))`` and + extrapolates a constant agreement rate beyond ``D``: + + RBO_EXT = (X_D / D) * p^D + + (1 - p) * sum_{d=1}^{D} p^(d-1) * X_d / d + + where ``X_d = |A_d ∩ B_d|`` is the intersection size at depth d. + This is the conventional "RBO score" used in IR practice and + satisfies the unit-bound property: identical equal-length lists + score 1.0, fully disjoint lists score 0.0. The non-extrapolated + RBO_MIN underestimates identical lists by ``p^D`` and was + rejected here because the tests in #796 R4 rely on identical → + 1.0 as a sanity gate. + + ``p`` controls top-weight: ``p → 0`` weights rank 1 only; + ``p → 1`` weights the tail almost as much as the head. The #796 + R4 finding used ``p = 0.9`` to give the top-K meaningful weight + without ignoring downstream rearrangements; that is the default. + Lists of unequal length are compared on their common prefix + length ``D``; tail items past ``D`` are ignored. + + Properties covered by the test suite: + * Identical lists → 1.0. + * Disjoint lists → 0.0. + * Monotone in prefix agreement: extending a shared prefix never + lowers the score. + * Both empty → 1.0 (vacuous identity); one empty → 0.0. + """ + if not 0.0 < p < 1.0: + raise ValueError("p must be in the open interval (0, 1)") + la, lb = len(a), len(b) + if la == 0 and lb == 0: + return 1.0 + if la == 0 or lb == 0: + return 0.0 + depth = min(la, lb) + seen_a: set[object] = set() + seen_b: set[object] = set() + overlap_count = 0 + weighted_sum = 0.0 + final_agreement = 0.0 + for d in range(depth): + x = a[d] + if x in seen_b: + overlap_count += 1 + seen_a.add(x) + y = b[d] + if y in seen_a: + overlap_count += 1 + seen_b.add(y) + agreement = overlap_count / (d + 1) + weighted_sum += (p ** d) * agreement + final_agreement = agreement + # RBO_EXT extrapolation term: assume agreement stays at the depth-D + # rate for ranks beyond D. For identical equal-length lists this + # term equals p^D so the total converges to 1.0. + return final_agreement * (p ** depth) + (1.0 - p) * weighted_sum diff --git a/src/aelfrice/eval_harness.py b/src/aelfrice/eval_harness.py index e2477cb81..ce6ea8d85 100644 --- a/src/aelfrice/eval_harness.py +++ b/src/aelfrice/eval_harness.py @@ -30,7 +30,9 @@ from aelfrice.calibration_metrics import ( CalibrationReport, + ordered_top_k_overlap, precision_at_k, + rank_biased_overlap, roc_auc, spearman_rho, ) @@ -42,12 +44,23 @@ "DEFAULT_CALIBRATION_CORPUS", "DEFAULT_K", "DEFAULT_SEED", + "DEFAULT_RBO_PERSISTENCE", "load_calibration_fixtures", "build_calibration_store", "run_calibration_on_fixtures", "format_calibration_report", + "RankComparisonReport", + "compare_ranking_panel", + "format_ranking_comparison", ) +# #796 R4 default — top-weight that gives the top-K meaningful weight +# without ignoring tail rearrangements. Chosen so a single transposition +# at rank 1 shifts the score noticeably while a transposition at rank +# 10 does not. Held in this module so the γ-vs-log-additive A/B bench +# and any later harness see the same default. +DEFAULT_RBO_PERSISTENCE: float = 0.9 + DEFAULT_CALIBRATION_CORPUS = ( Path(__file__).resolve().parent.parent.parent / "benchmarks" @@ -232,3 +245,93 @@ def format_calibration_report( lines.append(f"ROC-AUC: {_format_optional_float(report.roc_auc)}") lines.append(f"Spearman ρ: {_format_optional_float(report.spearman_rho)}") return "\n".join(lines) + "\n" + + +# --------------------------------------------------------------------------- +# #796 R4 rank-comparison panel +# --------------------------------------------------------------------------- +# When comparing two retrieval configurations (γ rerank vs log-additive +# baseline; bm25 vs bm25f; etc.) on the same query corpus, PR@K and +# Spearman ρ measure each config independently against the relevance +# label set. They cannot tell whether the configs produce the same +# top-K order or merely the same top-K set. The R4 campaign added +# ``ordered_top_k_overlap`` and ``rank_biased_overlap`` to discriminate +# top-of-list churn from middle-of-list reorderings; this panel +# averages those metrics across queries and exposes them in a stable +# report shape. The γ-vs-log-additive bench harness is the load- +# bearing consumer. + + +from dataclasses import dataclass # noqa: E402 — keep grouping with #796 block + + +@dataclass(frozen=True) +class RankComparisonReport: + """Per-query mean of rank-overlap metrics between two retrieval + configurations on a shared query set. + + ``ordered_top_k`` is the mean of ``ordered_top_k_overlap(a_i, b_i, k)`` + across queries. ``rbo`` is the mean of + ``rank_biased_overlap(a_i, b_i, p)``. Both are bounded in [0, 1]; + 1.0 means the two configs agree at the measured granularity, 0.0 + means they disagree completely. + + ``n_queries`` is the number of (a, b) pairs aggregated. Queries + where both rankings are empty contribute 1.0 to RBO (vacuous + identity) but 0.0 to ordered_top_k (no positions to match). + """ + + k: int + p: float + n_queries: int + ordered_top_k: float + rbo: float + + +def compare_ranking_panel( + pairs: Sequence[tuple[Sequence[object], Sequence[object]]], + *, + k: int = DEFAULT_K, + p: float = DEFAULT_RBO_PERSISTENCE, +) -> RankComparisonReport: + """Compute the #796 rank-comparison panel for paired rankings. + + Each pair ``(a_i, b_i)`` is the ranked-id list from configuration A + and configuration B on the same query. The two metrics are computed + per pair and arithmetic-mean-aggregated across pairs. An empty + ``pairs`` list raises ``ValueError`` (no meaningful average). + """ + if not pairs: + raise ValueError("pairs must be non-empty") + if k <= 0: + raise ValueError("k must be positive") + otk_vals: list[float] = [] + rbo_vals: list[float] = [] + for a, b in pairs: + otk_vals.append(ordered_top_k_overlap(a, b, k)) + rbo_vals.append(rank_biased_overlap(a, b, p=p)) + return RankComparisonReport( + k=k, + p=p, + n_queries=len(pairs), + ordered_top_k=sum(otk_vals) / len(otk_vals), + rbo=sum(rbo_vals) / len(rbo_vals), + ) + + +def format_ranking_comparison(report: RankComparisonReport) -> str: + """Format a ``RankComparisonReport`` as a deterministic text block. + + Mirrors ``format_calibration_report``'s shape so the two panels + sit cleanly side-by-side in a combined bench run. + """ + lines = [ + "rank-comparison panel — γ vs log-additive (#796 R4)", + f" n_queries: {report.n_queries}", + f" k: {report.k}", + f" p: {report.p:.4f}", + "", + f"ordered_top_k@{report.k}: {report.ordered_top_k:.4f}", + f"RBO(p={report.p:.2f}): {report.rbo:.4f}", + ] + return "\n".join(lines) + "\n" diff --git a/src/aelfrice/retrieval.py b/src/aelfrice/retrieval.py index d2c79d4c4..c088df7bc 100644 --- a/src/aelfrice/retrieval.py +++ b/src/aelfrice/retrieval.py @@ -98,6 +98,7 @@ from aelfrice.models import LOCK_NONE, LOCK_USER, Belief from aelfrice.scoring import ( DEFAULT_POSTERIOR_WEIGHT, + gamma_posterior_score, partial_bayesian_score, posterior_mean, ) @@ -166,6 +167,16 @@ # AELFRICE_INTENTIONAL_CLUSTERING=0, or # `[retrieval] use_intentional_clustering = false`. INTENTIONAL_CLUSTERING_FLAG: Final[str] = "use_intentional_clustering" +# v3.x #796 γ rerank flag. Default-OFF until a labeled relevance corpus +# exists and the bench panel (PR@5 + ρ + ordered_top_k_overlap + +# rank_biased_overlap) demonstrates uplift over log-additive. When ON, +# `_l1_hits` routes its rerank through `gamma_posterior_score(...)` with +# `T = resolve_posterior_temperature_with_meta(...)` (defaults to 1.0 +# when the meta-belief is absent — byte-identical to +# `partial_bayesian_score` at `posterior_weight = 1.0`). +USE_GAMMA_POSTERIOR_TEMPERATURE_FLAG: Final[str] = ( + "use_gamma_posterior_temperature" +) PLACEHOLDER_FLAGS: Final[tuple[str, ...]] = ( SIGNED_LAPLACIAN_FLAG, @@ -199,6 +210,10 @@ ENV_TYPE_AWARE_COMPRESSION: Final[str] = "AELFRICE_TYPE_AWARE_COMPRESSION" # v2.0 #436 intentional-clustering env override. Tri-state. ENV_INTENTIONAL_CLUSTERING: Final[str] = "AELFRICE_INTENTIONAL_CLUSTERING" +# v3.x #796 γ rerank env override. Tri-state, default-OFF. +ENV_USE_GAMMA_POSTERIOR_TEMPERATURE: Final[str] = ( + "AELFRICE_USE_GAMMA_POSTERIOR_TEMPERATURE" +) # v1.3.0 posterior-weight env override. Float-typed; "0.0" is the # only value that fully disables (collapsing to BM25-only ordering). # Empty / non-numeric values fall through to the next precedence @@ -338,6 +353,28 @@ ENV_META_BELIEF_EXPANSION_GATE_TOKEN_THRESHOLD: Final[str] = ( "AELFRICE_META_BELIEF_EXPANSION_GATE_TOKEN_THRESHOLD" ) +# --------------------------------------------------------------------------- +# #796 γ-rerank posterior-temperature meta-belief consumer +# --------------------------------------------------------------------------- +# Boltzmann temperature `T` on the posterior log term, consumed by +# `scoring.gamma_posterior_score`. Bounds `[0.5, 2.0]`; geometric mean +# is exactly 1.0, so the cold-start decode at `static_default=0.5` is +# `T = 1.0` — byte-identical to `partial_bayesian_score` with +# `posterior_weight = 1.0`. Adaptive learning of `T` (the #758 follow- +# up) is out of scope for #796: this issue ships the surface and the +# default-OFF flag, and the bench panel records γ vs log-additive +# under a hardcoded `T = 1.0`. The meta-belief substrate is installed +# here so the #758 wiring can drop in without a second config flip. +META_POSTERIOR_TEMPERATURE_KEY: Final[str] = ( + "meta:retrieval.posterior_temperature" +) +POSTERIOR_TEMPERATURE_FLOOR: Final[float] = 0.5 +POSTERIOR_TEMPERATURE_CEIL: Final[float] = 2.0 +# Mid-range so cold-start decodes to `T = 1.0` exactly. +META_POSTERIOR_TEMPERATURE_STATIC_DEFAULT: Final[float] = 0.5 +# Sub-posterior decay — 30d, matching the rest of the #480 family. +META_POSTERIOR_TEMPERATURE_POSTERIOR_DECAY_SECONDS: Final[int] = 30 * 24 * 3600 + # Number of decimal places used to round `posterior_weight` before # inclusion in the cache key. Two callers passing weights that # differ by less than this granularity collapse to the same key. @@ -556,6 +593,21 @@ def _env_intentional_clustering_override() -> bool | None: return None +def _env_use_gamma_posterior_temperature_override() -> bool | None: + """Return True/False if AELFRICE_USE_GAMMA_POSTERIOR_TEMPERATURE is + set to a recognised truthy/falsy value, else None. Symmetric to + `_env_type_aware_compression_override`.""" + raw = os.environ.get(ENV_USE_GAMMA_POSTERIOR_TEMPERATURE) + if raw is None: + return None + norm = raw.strip().lower() + if norm in _ENV_FALSY: + return False + if norm in _ENV_TRUTHY: + return True + return None + + def _env_hrr_persist_override() -> bool | None: """Return True/False if AELFRICE_HRR_PERSIST is set to a recognised truthy/falsy value, else None. Symmetric to `_env_bm25f_override`. @@ -1635,6 +1687,83 @@ def resolve_use_intentional_clustering( return True +def resolve_use_gamma_posterior_temperature( + explicit: bool | None = None, + *, + start: Path | None = None, +) -> bool: + """Resolve the γ rerank flag (#796). + + Precedence (first decisive wins): + 1. AELFRICE_USE_GAMMA_POSTERIOR_TEMPERATURE env var. + 2. Explicit `explicit` kwarg from the caller. + 3. `[retrieval] use_gamma_posterior_temperature` in `.aelfrice.toml`. + 4. Default: False — ships behind the flag at v3.x. Adoption verdict + (flip default) is deferred until a labeled relevance corpus + exists, per `docs/feature-posterior-temperature.md` § + "Bench-gate / ship-or-defer policy". + """ + env = _env_use_gamma_posterior_temperature_override() + if env is not None: + return env + if explicit is not None: + return explicit + toml_value = _read_toml_flag_for( + USE_GAMMA_POSTERIOR_TEMPERATURE_FLAG, start, + ) + if toml_value is not None: + return toml_value + return False + + +def resolve_posterior_temperature_with_meta( + store: "MemoryStore | None", + *, + now_ts: int, +) -> float: + """Resolve the γ-rerank Boltzmann temperature `T` (#796). + + Reads `meta:retrieval.posterior_temperature` from the store via + `read_meta_belief_value`. Returns: + + * `T = 1.0` when `store` is None or the meta-belief is not + installed — the byte-identical-to-log-additive case (γ at + `T = 1.0` equals `partial_bayesian_score(..., 1.0)`). + * Log-linear decode of the meta-belief value to + `[POSTERIOR_TEMPERATURE_FLOOR, POSTERIOR_TEMPERATURE_CEIL]` + otherwise. With the static_default of 0.5 the decode lands + at the geometric mean 1.0 exactly, so a cold-start install + is still byte-identical until evidence accumulates. + + Adaptive learning of `T` is #758's scope; #796 ships the surface + and the decoder only. The store read is best-effort — any error + falls back to `T = 1.0` rather than raising. + """ + if store is None: + return 1.0 + try: + raw = store.read_meta_belief_value( + META_POSTERIOR_TEMPERATURE_KEY, now_ts=now_ts, + ) + except Exception as exc: # noqa: BLE001 + print( + "aelfrice retrieval: posterior-temperature meta-belief " + f"read failed: {exc}", + file=sys.stderr, + ) + return 1.0 + if raw is None: + return 1.0 + # Clamp the [0, 1] posterior surface value to its valid band before + # the log-linear decode. The store contract should already keep it + # in-range; the clamp is defensive against future code paths that + # write raw values. + v = max(0.0, min(1.0, float(raw))) + log_floor = math.log(POSTERIOR_TEMPERATURE_FLOOR) + log_ceil = math.log(POSTERIOR_TEMPERATURE_CEIL) + return math.exp(log_floor + v * (log_ceil - log_floor)) + + _PLACEHOLDER_WARNED: set[str] = set() @@ -1885,6 +2014,7 @@ def _l1_hits( bm25f_cache: BM25IndexCache | None = None, eigenbasis_cache: GraphEigenbasisCache | None = None, heat_kernel_on: bool = False, + gamma_temperature: float | None = None, ) -> list[Belief]: """Run L1: FTS5 BM25 search (default) or BM25F sparse-matvec (v1.5.0 opt-in), optionally reranked by partial-Bayesian score. @@ -1903,6 +2033,17 @@ def _l1_hits( `posterior_weight > 0` reranks via `partial_bayesian_score`. + `gamma_temperature` (v3.x #796): when not None AND `heat_kernel_on` + is False (or no eigenbasis is available), the rerank loop swaps + `partial_bayesian_score(bm25_raw, α, β, posterior_weight)` for + `gamma_posterior_score(bm25_raw, α, β, gamma_temperature)`. At + `T = 1.0` γ is byte-identical to `partial_bayesian_score` at + `posterior_weight = 1.0`. None falls through to the log-additive + path. When `heat_kernel_on` is True and the heat-rerank fires γ is + a no-op on this call — the two scoring paths are mutually + exclusive by design (the operator decision deferred composition + to a later issue). + `heat_kernel_on` (v1.7.0): when True AND `eigenbasis_cache` holds a non-stale eigenbasis whose `belief_ids` intersect the L1 hit set, the rerank uses `combine_log_scores(bm25, heat, posterior_mean)` @@ -1986,7 +2127,15 @@ def _l1_hits( if b is None: continue beliefs.append((b, raw)) - if posterior_weight == 0.0 and not heat_active and not hash_n_literals: + # γ is opt-in; when set it forces the rerank loop so the + # byte-identical short-circuit can't bypass the temperature + # reweighting. + if ( + posterior_weight == 0.0 + and not heat_active + and not hash_n_literals + and gamma_temperature is None + ): return [b for b, _ in beliefs] # BM25F scores are non-negative; the rerank uses `raw` as the # positive-magnitude relevance signal directly (the FTS5 path @@ -2010,6 +2159,10 @@ def _l1_hits( else DEFAULT_POSTERIOR_LOG_WEIGHT ), ) + elif gamma_temperature is not None: + s = gamma_posterior_score( + -raw, b.alpha, b.beta, gamma_temperature, + ) else: s = partial_bayesian_score( -raw, b.alpha, b.beta, posterior_weight, @@ -2019,7 +2172,12 @@ def _l1_hits( keyed.sort(key=lambda x: (-x[0], x[1])) return [b for _, _, b in keyed] - if posterior_weight == 0.0 and not heat_active and not hash_n_literals: + if ( + posterior_weight == 0.0 + and not heat_active + and not hash_n_literals + and gamma_temperature is None + ): return store.search_beliefs(query, limit=l1_limit) scored = store.search_beliefs_scored(query, limit=l1_limit) if not scored: @@ -2045,6 +2203,10 @@ def _l1_hits( else DEFAULT_POSTERIOR_LOG_WEIGHT ), ) + elif gamma_temperature is not None: + s = gamma_posterior_score( + bm25_raw, b.alpha, b.beta, gamma_temperature, + ) else: s = partial_bayesian_score( bm25_raw, b.alpha, b.beta, posterior_weight, @@ -2134,6 +2296,18 @@ def retrieve( bm25f_on = resolve_use_bm25f_anchors(use_bm25f_anchors) weight = resolve_posterior_weight(posterior_weight) heat_on = is_heat_kernel_enabled(heat_kernel_enabled) + # #796 γ rerank — resolve once per call so the temperature read + # is consistent across BM25F / FTS5 branches and the audit-trail + # sees a single store hit. When the flag is off, gamma_t stays + # None and `_l1_hits` skips the γ branch entirely (byte-identical + # to the log-additive baseline). + gamma_on = resolve_use_gamma_posterior_temperature() + gamma_t = ( + resolve_posterior_temperature_with_meta( + store, now_ts=int(time.time()), + ) + if gamma_on else None + ) compress_on = resolve_use_type_aware_compression( use_type_aware_compression, ) @@ -2211,6 +2385,7 @@ def _cost(b: Belief) -> int: l1_limit=l1_limit, posterior_weight=weight, use_bm25f_anchors=bm25f_on, bm25f_cache=bm25f_cache, eigenbasis_cache=eigenbasis_cache, heat_kernel_on=heat_on, + gamma_temperature=gamma_t, ) l1 = [ b for b in raw_l1 @@ -2344,6 +2519,14 @@ def retrieve_with_tiers( bm25f_on = resolve_use_bm25f_anchors(use_bm25f_anchors) weight = resolve_posterior_weight(posterior_weight) heat_on = is_heat_kernel_enabled(heat_kernel_enabled) + # #796 γ rerank — same resolution as `retrieve()`. + gamma_on = resolve_use_gamma_posterior_temperature() + gamma_t = ( + resolve_posterior_temperature_with_meta( + store, now_ts=int(time.time()), + ) + if gamma_on else None + ) # #741 adaptive expansion-gate. Same shape as retrieve(): short- # circuit BFS on broad prompts; L0 / L1 / L2.5-entity unaffected. # #760: pass store + now_ts for the meta-belief token-threshold @@ -2414,6 +2597,7 @@ def _cost(b: Belief) -> int: l1_limit=l1_limit, posterior_weight=weight, use_bm25f_anchors=bm25f_on, bm25f_cache=bm25f_cache, eigenbasis_cache=eigenbasis_cache, heat_kernel_on=heat_on, + gamma_temperature=gamma_t, ) l1 = [ b for b in raw_l1 diff --git a/src/aelfrice/scoring.py b/src/aelfrice/scoring.py index 2e6e17e88..a13b4e7e1 100644 --- a/src/aelfrice/scoring.py +++ b/src/aelfrice/scoring.py @@ -49,6 +49,12 @@ # λ=0.5; collapses to 0.91 at λ=1.0; minimal effect at λ=0.0). DEFAULT_POSTERIOR_WEIGHT: Final[float] = 0.5 +# #796 γ rerank — minimum temperature accepted by +# `gamma_posterior_score`. T must be strictly positive (division); +# values below this floor clamp upward so a misconfigured meta-belief +# or env override never raises at retrieval time. +GAMMA_TEMPERATURE_FLOOR: Final[float] = 1e-6 + # --- Half-lives in seconds --- _HOUR: Final[float] = 3600.0 TYPE_HALF_LIFE_SECONDS: Final[dict[str, float]] = { @@ -201,3 +207,36 @@ def partial_bayesian_score( # pathological `alpha = 0` operator-fed case to avoid `log(0)`. p_safe = p if p > 0.0 else PARTIAL_BAYESIAN_BM25_FLOOR return log_bm25 + posterior_weight * math.log(p_safe) + + +def gamma_posterior_score( + bm25_raw: float, + alpha: float, + beta: float, + temperature: float, +) -> float: + """#796 γ rerank — Boltzmann temperature on the posterior log term. + + `score = log(max(-bm25_raw, EPS)) + (1 / T) * log(posterior_mean)` + + At `T = 1.0` this collapses to `partial_bayesian_score` with + `posterior_weight = 1.0` (byte-identical). Lower `T` sharpens the + posterior contribution (high-posterior beliefs pull harder); + higher `T` flattens it toward BM25-only ranking. + + Temperatures at or below `GAMMA_TEMPERATURE_FLOOR` clamp upward — + a misconfigured meta-belief value never raises at retrieval time. + Negative temperatures are likewise clamped: the Boltzmann reading + is undefined for `T <= 0` and the safest fall-back is the floor. + + γ is the load-bearing precursor to #758's adaptive + `meta:retrieval.posterior_temperature`. Until that meta-belief is + populated and learning, callers pin `T = 1.0` and the bench panel + measures the γ vs log-additive surface (#796 R&D campaign verdict). + """ + t_safe = temperature if temperature > GAMMA_TEMPERATURE_FLOOR else ( + GAMMA_TEMPERATURE_FLOOR + ) + return partial_bayesian_score( + bm25_raw, alpha, beta, posterior_weight=(1.0 / t_safe), + ) diff --git a/tests/test_rank_overlap_metrics.py b/tests/test_rank_overlap_metrics.py new file mode 100644 index 000000000..f7a888948 --- /dev/null +++ b/tests/test_rank_overlap_metrics.py @@ -0,0 +1,170 @@ +"""Tests for #796 R4 rank-overlap metrics — ordered_top_k_overlap and +rank_biased_overlap. + +Properties under test (per the operator-mandated acceptance): + +* ``ordered_top_k_overlap`` is 1.0 on identical prefixes and 0.0 on a + full reverse of an even-length prefix. Linear in the count of + position-wise matches. +* ``rank_biased_overlap`` (RBO_EXT) is 1.0 on identical equal-length + lists, 0.0 on disjoint lists, and monotone non-decreasing as the + shared prefix length grows. +* Both are deterministic across calls. +* Empty-list edge cases hold (both empty → 1.0 for RBO, 0.0 for + ordered_top_k; one empty → 0.0 for RBO). + +These two metrics widen the eval panel so the γ-vs-log-additive +bench (and any future A/B retrieval comparison) can discriminate top- +of-list churn from middle-of-list reorderings — the R4 finding that +PR@K + Spearman ρ alone cannot. +""" +from __future__ import annotations + +import pytest + +from aelfrice.calibration_metrics import ( + ordered_top_k_overlap, + rank_biased_overlap, +) +from aelfrice.eval_harness import ( + DEFAULT_RBO_PERSISTENCE, + compare_ranking_panel, + format_ranking_comparison, +) + + +# --------------------------------------------------------------------------- +# ordered_top_k_overlap +# --------------------------------------------------------------------------- + +def test_otk_identical_prefix_is_one() -> None: + assert ordered_top_k_overlap([1, 2, 3], [1, 2, 3], 3) == 1.0 + + +def test_otk_reversed_even_prefix_is_zero() -> None: + """An even-length full reverse has no position-wise match.""" + assert ordered_top_k_overlap([1, 2, 3, 4], [4, 3, 2, 1], 4) == 0.0 + + +def test_otk_reversed_odd_prefix_keeps_middle() -> None: + """An odd-length reverse fixes the middle element; score = 1/k.""" + assert ordered_top_k_overlap([1, 2, 3], [3, 2, 1], 3) == 1.0 / 3.0 + + +def test_otk_disjoint_prefix_is_zero() -> None: + assert ordered_top_k_overlap([1, 2, 3], [4, 5, 6], 3) == 0.0 + + +def test_otk_shorter_input_counts_missing_as_mismatch() -> None: + assert ordered_top_k_overlap([1, 2, 3], [], 3) == 0.0 + assert ordered_top_k_overlap([1, 2, 3], [1], 3) == pytest.approx(1.0 / 3.0) + + +def test_otk_invalid_k_raises() -> None: + with pytest.raises(ValueError): + ordered_top_k_overlap([1, 2, 3], [1, 2, 3], 0) + with pytest.raises(ValueError): + ordered_top_k_overlap([1, 2, 3], [1, 2, 3], -1) + + +def test_otk_linear_in_matches() -> None: + """k=4, three matching positions → 3/4.""" + assert ordered_top_k_overlap( + [1, 2, 3, 4], [1, 2, 3, 99], 4, + ) == pytest.approx(0.75) + + +# --------------------------------------------------------------------------- +# rank_biased_overlap (RBO_EXT) +# --------------------------------------------------------------------------- + +def test_rbo_identical_equal_length_is_one() -> None: + assert rank_biased_overlap([1, 2, 3, 4, 5], [1, 2, 3, 4, 5]) == pytest.approx(1.0) + # Single-element identical also caps at 1.0 + assert rank_biased_overlap([1], [1]) == pytest.approx(1.0) + + +def test_rbo_disjoint_is_zero() -> None: + assert rank_biased_overlap([1, 2, 3], [4, 5, 6]) == 0.0 + + +def test_rbo_both_empty_is_one() -> None: + assert rank_biased_overlap([], []) == 1.0 + + +def test_rbo_one_empty_is_zero() -> None: + assert rank_biased_overlap([], [1, 2, 3]) == 0.0 + assert rank_biased_overlap([1, 2, 3], []) == 0.0 + + +def test_rbo_monotone_in_prefix_agreement() -> None: + """Extending a shared prefix never lowers the RBO score.""" + no_overlap = rank_biased_overlap([1, 2, 3], [9, 9, 9]) + one_overlap = rank_biased_overlap([1, 2, 3], [1, 9, 9]) + two_overlap = rank_biased_overlap([1, 2, 3], [1, 2, 9]) + three_overlap = rank_biased_overlap([1, 2, 3], [1, 2, 3]) + assert no_overlap <= one_overlap <= two_overlap <= three_overlap + assert three_overlap == pytest.approx(1.0) + + +def test_rbo_top_swap_costs_more_than_tail_swap() -> None: + """A swap at rank 1 hurts RBO more than the same swap deep in the list. + + This is the load-bearing property — γ vs log-additive can have + identical PR@5 + ρ but different top-K ordering, and RBO at + p=0.9 picks that up. + """ + base = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] + top_swap = [2, 1, 3, 4, 5, 6, 7, 8, 9, 10] + tail_swap = [1, 2, 3, 4, 5, 6, 7, 8, 10, 9] + top_score = rank_biased_overlap(base, top_swap, p=0.9) + tail_score = rank_biased_overlap(base, tail_swap, p=0.9) + assert top_score < tail_score < 1.0 + + +def test_rbo_invalid_p_raises() -> None: + with pytest.raises(ValueError): + rank_biased_overlap([1], [1], p=0.0) + with pytest.raises(ValueError): + rank_biased_overlap([1], [1], p=1.0) + + +def test_rbo_deterministic_across_calls() -> None: + a = [1, 2, 3, 4, 5] + b = [1, 3, 2, 5, 4] + first = rank_biased_overlap(a, b, p=0.9) + second = rank_biased_overlap(a, b, p=0.9) + assert first == second + + +# --------------------------------------------------------------------------- +# Eval-harness panel +# --------------------------------------------------------------------------- + +def test_compare_ranking_panel_averages_correctly() -> None: + pairs = [ + ([1, 2, 3], [1, 2, 3]), # identical + ([1, 2, 3], [3, 2, 1]), # reversed + ] + r = compare_ranking_panel(pairs, k=3, p=DEFAULT_RBO_PERSISTENCE) + # ordered_top_k mean: (1.0 + 1/3) / 2 = 2/3 + assert r.ordered_top_k == pytest.approx(2.0 / 3.0) + # RBO mean: (1.0 + rbo([1,2,3],[3,2,1])) / 2; just bound-check. + assert 0.0 <= r.rbo <= 1.0 + assert r.n_queries == 2 + assert r.k == 3 + assert r.p == DEFAULT_RBO_PERSISTENCE + + +def test_compare_ranking_panel_empty_pairs_raises() -> None: + with pytest.raises(ValueError): + compare_ranking_panel([], k=3) + + +def test_format_ranking_comparison_is_stable_text() -> None: + pairs = [([1, 2, 3], [1, 2, 3])] + r = compare_ranking_panel(pairs, k=3, p=0.9) + text = format_ranking_comparison(r) + assert "rank-comparison panel" in text + assert "n_queries: 1" in text + assert "ordered_top_k@3: 1.0000" in text diff --git a/tests/test_retrieve_gamma_flag.py b/tests/test_retrieve_gamma_flag.py new file mode 100644 index 000000000..efcafca5e --- /dev/null +++ b/tests/test_retrieve_gamma_flag.py @@ -0,0 +1,163 @@ +"""Tests for #796 γ flag wiring in retrieve_v2 / retrieve_with_tiers. + +Properties under test: + +1. **Flag-off byte-identity.** With ``AELFRICE_USE_GAMMA_POSTERIOR_TEMPERATURE`` + unset (the default), retrieve()'s output is unchanged compared to a + pre-#796 baseline — the existing log-additive contract holds. +2. **Flag-on, meta-belief absent.** The resolver returns T=1.0 + (byte-identical to ``partial_bayesian_score(.., 1.0)``), so γ runs + the rerank loop but its score is anchored to the known log-additive + reference. Output is deterministic given the same store + query. +3. **Resolver precedence.** env > kwarg > TOML > False. Verified by + the resolver-only tests (no store touch needed for the precedence + chain). +4. **Temperature decoder bounds.** ``resolve_posterior_temperature_with_meta`` + returns 1.0 on a None store, decodes log-linearly into + ``[POSTERIOR_TEMPERATURE_FLOOR, POSTERIOR_TEMPERATURE_CEIL]``, and + hits exactly 1.0 at the static-default mid-value of 0.5. +""" +from __future__ import annotations + +import math +import uuid + +import pytest + +from aelfrice.models import BELIEF_FACTUAL, LOCK_NONE, RETENTION_FACT, Belief +from aelfrice.retrieval import ( + POSTERIOR_TEMPERATURE_CEIL, + POSTERIOR_TEMPERATURE_FLOOR, + resolve_posterior_temperature_with_meta, + resolve_use_gamma_posterior_temperature, + retrieve, +) +from aelfrice.store import MemoryStore + + +_ENV_FLAG = "AELFRICE_USE_GAMMA_POSTERIOR_TEMPERATURE" + + +@pytest.fixture(autouse=True) +def _isolate_env(monkeypatch): + """Ensure the γ env flag never leaks across tests.""" + monkeypatch.delenv(_ENV_FLAG, raising=False) + yield + + +def _mk_belief(text: str, *, alpha: float = 1.0, beta: float = 1.0) -> Belief: + bid = uuid.uuid4().hex[:16] + return Belief( + id=bid, + content=text, + content_hash=f"h_{bid}", + alpha=alpha, + beta=beta, + type=BELIEF_FACTUAL, + lock_level=LOCK_NONE, + locked_at=None, + created_at="2023-11-14T22:13:20+00:00", + last_retrieved_at=None, + retention_class=RETENTION_FACT, + ) + + +@pytest.fixture +def populated_store(): + """Fresh in-memory store with a small corpus that exercises + posterior reweighting (the two ``letter`` beliefs have different + α/β so γ can move them relative to each other).""" + s = MemoryStore(":memory:") + s.insert_belief(_mk_belief("alpha is the first letter", alpha=10.0, beta=1.0)) + s.insert_belief(_mk_belief("beta is the second letter", alpha=1.0, beta=10.0)) + s.insert_belief(_mk_belief("gamma is the third letter", alpha=5.0, beta=5.0)) + s.insert_belief(_mk_belief("delta is the fourth letter", alpha=1.0, beta=1.0)) + yield s + s.close() + + +# --------------------------------------------------------------------------- +# Resolver precedence +# --------------------------------------------------------------------------- + +def test_resolver_default_false(monkeypatch) -> None: + monkeypatch.delenv(_ENV_FLAG, raising=False) + assert resolve_use_gamma_posterior_temperature() is False + + +def test_resolver_env_truthy_wins(monkeypatch) -> None: + monkeypatch.setenv(_ENV_FLAG, "1") + assert resolve_use_gamma_posterior_temperature() is True + + +def test_resolver_env_falsy_wins_over_kwarg(monkeypatch) -> None: + monkeypatch.setenv(_ENV_FLAG, "0") + assert resolve_use_gamma_posterior_temperature(explicit=True) is False + + +def test_resolver_explicit_kwarg_when_env_unset(monkeypatch) -> None: + monkeypatch.delenv(_ENV_FLAG, raising=False) + assert resolve_use_gamma_posterior_temperature(explicit=True) is True + assert resolve_use_gamma_posterior_temperature(explicit=False) is False + + +def test_resolver_unrecognised_env_falls_through(monkeypatch) -> None: + monkeypatch.setenv(_ENV_FLAG, "maybe") + assert resolve_use_gamma_posterior_temperature() is False + + +# --------------------------------------------------------------------------- +# Temperature decoder +# --------------------------------------------------------------------------- + +def test_temperature_decoder_none_store_returns_one() -> None: + assert resolve_posterior_temperature_with_meta(None, now_ts=0) == 1.0 + + +def test_temperature_decoder_static_default_geometric_mean() -> None: + """Manually verify the log-linear decode at v=0.5 lands at T=1.0 + (the documented byte-identical contract). This guards against + accidental bound changes that would break the cold-start + invariant.""" + log_floor = math.log(POSTERIOR_TEMPERATURE_FLOOR) + log_ceil = math.log(POSTERIOR_TEMPERATURE_CEIL) + decoded = math.exp(log_floor + 0.5 * (log_ceil - log_floor)) + assert math.isclose(decoded, 1.0, abs_tol=1e-12) + + +# --------------------------------------------------------------------------- +# Flag-off byte-identity + flag-on determinism +# --------------------------------------------------------------------------- + +def test_flag_off_baseline_unchanged(populated_store, monkeypatch) -> None: + """Flag unset → retrieve() produces the same output across calls + (the regression-protection lane).""" + monkeypatch.delenv(_ENV_FLAG, raising=False) + a = retrieve(populated_store, "letter alphabet") + b = retrieve(populated_store, "letter alphabet") + assert [x.id for x in a] == [x.id for x in b] + + +def test_flag_on_deterministic(populated_store, monkeypatch) -> None: + """Flag-on, no meta-belief → T=1.0 → output is deterministic + across repeats.""" + monkeypatch.setenv(_ENV_FLAG, "1") + a = retrieve(populated_store, "letter alphabet") + b = retrieve(populated_store, "letter alphabet") + assert [x.id for x in a] == [x.id for x in b] + + +def test_flag_on_versus_off_runs_clean(populated_store, monkeypatch) -> None: + """Sanity: flipping the flag on does not raise on a small corpus. + + This is the smoke gate the broader bench-corpus comparison sits + on top of. The actual γ-vs-log-additive ranking comparison is the + job of the lab-side A/B harness, not this unit test. + """ + monkeypatch.delenv(_ENV_FLAG, raising=False) + off = retrieve(populated_store, "letter alphabet") + monkeypatch.setenv(_ENV_FLAG, "1") + on = retrieve(populated_store, "letter alphabet") + # Both produced something; the content / order is the bench's job. + assert isinstance(off, list) + assert isinstance(on, list) diff --git a/tests/test_scoring_gamma.py b/tests/test_scoring_gamma.py new file mode 100644 index 000000000..b7e729a77 --- /dev/null +++ b/tests/test_scoring_gamma.py @@ -0,0 +1,98 @@ +"""Tests for #796 γ rerank — gamma_posterior_score. + +Properties under test: + +1. **T=1.0 byte-identity.** ``gamma_posterior_score(bm, α, β, 1.0)`` is + bit-for-bit equal to ``partial_bayesian_score(bm, α, β, 1.0)`` — + the operator-mandated reference. The bench panel uses this to + anchor the γ surface against a known log-additive baseline. +2. **Reciprocal-temperature equivalence.** For any T > 0, + ``gamma_posterior_score(bm, α, β, T)`` equals + ``partial_bayesian_score(bm, α, β, 1/T)``. This is the load-bearing + contract — γ is reparametrised log-additive, not a new function + family. +3. **Floor clamp.** Non-positive temperatures clamp upward to + ``GAMMA_TEMPERATURE_FLOOR`` rather than raising. Operationally this + means a misconfigured meta-belief or env override degrades to a + very-sharp posterior weighting instead of crashing retrieval. +4. **Determinism.** Same inputs → same float bits across calls. +""" +from __future__ import annotations + +import math + +import pytest + +from aelfrice.scoring import ( + GAMMA_TEMPERATURE_FLOOR, + gamma_posterior_score, + partial_bayesian_score, +) + + +@pytest.mark.parametrize( + "bm25_raw,alpha,beta", + [ + (-1.5, 4.0, 2.0), + (-0.001, 0.5, 0.5), # Jeffreys prior, weak BM25 hit + (-10.0, 100.0, 1.0), # strong evidence, strong match + (0.0, 1.0, 1.0), # no-match (BM25 = 0); floor protects log + ], +) +def test_t_one_byte_identical_to_partial_bayesian_pw_one( + bm25_raw: float, alpha: float, beta: float, +) -> None: + """At T=1.0, γ collapses to partial_bayesian(posterior_weight=1.0).""" + g = gamma_posterior_score(bm25_raw, alpha, beta, 1.0) + p = partial_bayesian_score(bm25_raw, alpha, beta, 1.0) + assert g == p, f"γ({bm25_raw},{alpha},{beta},T=1)={g!r} ≠ pb={p!r}" + + +@pytest.mark.parametrize("temperature", [0.25, 0.5, 1.0, 1.5, 2.0, 5.0]) +def test_reciprocal_temperature_equivalence(temperature: float) -> None: + """γ(bm, α, β, T) == partial_bayesian(bm, α, β, 1/T) for any T > 0.""" + bm25_raw, alpha, beta = -1.5, 3.0, 2.0 + g = gamma_posterior_score(bm25_raw, alpha, beta, temperature) + p = partial_bayesian_score(bm25_raw, alpha, beta, 1.0 / temperature) + assert g == p, ( + f"T={temperature}: γ={g!r}, pb(pw={1.0 / temperature})={p!r}" + ) + + +def test_non_positive_temperature_clamps_to_floor() -> None: + """T <= 0 clamps to GAMMA_TEMPERATURE_FLOOR; never raises.""" + bm25_raw, alpha, beta = -1.5, 3.0, 2.0 + expected = partial_bayesian_score( + bm25_raw, alpha, beta, 1.0 / GAMMA_TEMPERATURE_FLOOR, + ) + for bad_t in [0.0, -0.5, -1e9]: + got = gamma_posterior_score(bm25_raw, alpha, beta, bad_t) + assert got == expected, f"T={bad_t}: got {got}, expected {expected}" + assert math.isfinite(got) + + +def test_higher_temperature_flattens_posterior_contribution() -> None: + """Hot beliefs ranked above cold ones; effect shrinks with T. + + With identical BM25 and α_hot=10, β_hot=1 vs α_cold=1, β_cold=10, + the score gap between the two should monotonically shrink as T + grows — high T flattens the posterior log term. + """ + bm = -1.5 + gaps = [] + for t in [0.5, 1.0, 2.0, 5.0]: + hot = gamma_posterior_score(bm, 10.0, 1.0, t) + cold = gamma_posterior_score(bm, 1.0, 10.0, t) + gaps.append(hot - cold) + assert all(gaps[i] > gaps[i + 1] for i in range(len(gaps) - 1)), gaps + # Both arms still favour the hot belief at every T. + for g in gaps: + assert g > 0 + + +def test_deterministic_across_calls() -> None: + """Same inputs → bit-identical output across calls.""" + for _ in range(3): + assert gamma_posterior_score(-1.5, 4.0, 2.0, 0.7) == ( + gamma_posterior_score(-1.5, 4.0, 2.0, 0.7) + )