Skip to content

feat(v1.3): partial Bayesian-weighted ranking — log-additive (closes #146) - #178

Merged
robotrocketscience merged 5 commits into
mainfrom
feat/bayesian-ranking-v1.3
Apr 28, 2026
Merged

feat(v1.3): partial Bayesian-weighted ranking — log-additive (closes #146)#178
robotrocketscience merged 5 commits into
mainfrom
feat/bayesian-ranking-v1.3

Conversation

@robotrocketscience

Copy link
Copy Markdown
Owner

Summary

Implements issue #146 — partial Bayesian-weighted ranking per the adopted Path B contract in docs/bayesian_ranking.md:

score = log(-bm25_raw) + posterior_weight * log(posterior_mean(α, β))
  • Default posterior_weight = 0.5 — synthetic-graph optimum from the v1.3 calibration. Configurable via [retrieval] posterior_weight in .aelfrice.toml and AELFRICE_POSTERIOR_WEIGHT=<float> env var. Negative values clamp to 0.0.
  • Byte-identical at posterior_weight = 0.0 — regression-tested against MemoryStore.search_beliefs directly (AC2) and against the size-16 benchmark corpus's 16 queries.
  • Reuses scoring.posterior_mean (Jeffreys prior α / (α+β)). The Laplace (α+1)/(α+β+2) form sketched in [retrieval] Posterior-weighted ranking via Beta-Bernoulli prior (log-additive, weight 0.5) #151 is explicitly rejected at this layer per the spec rationale.
  • Locks (L0) bypass scoring entirely. L2.5 entity-index hits and L3 BFS expansions are unaffected — the weight only reranks the L1 BM25 candidate set.
  • bm25 == 0 (FTS5 non-match) is floored at PARTIAL_BAYESIAN_BM25_FLOOR = 1e-12 so log(0) cannot raise.
  • RetrievalCache key gains posterior_weight (rounded to four decimals via POSTERIOR_WEIGHT_KEY_PRECISION) — additive, so existing cached entries with the old key shape are still functional but cannot collide with new-key entries. Cache invalidation is unchanged: apply_feedbackstore.update_belief_fire_invalidation → cache wipe. No new hook in apply_feedback.

Out of scope (per spec § "Out of scope")

10-round MRR uplift eval, ECE calibration scorer, BM25F + heat-kernel composition, real-feedback retest — all deferred to v2.0.0. Issue #151 should be re-scoped to that v2.0 follow-up per the spec's disposition recommendation.

README copy update is deferred to a follow-up docs sweep PR (orchestrator's Phase G).

Atomic commits

  1. feat(retrieval): add MemoryStore.search_beliefs_scored returning (belief, bm25) — exposes the FTS5 BM25 score; existing search_beliefs unchanged.
  2. feat(scoring): add partial_bayesian_score combining BM25 + posterior log-additively — pure scoring helper.
  3. feat(retrieval): wire posterior_weight into retrieve / cache keyretrieve() / retrieve_with_tiers() / retrieve_v2() plumbing + RetrievalCache key extension.
  4. test: 22 acceptance tests for partial Bayesian-weighted ranking (#146) — one test per spec acceptance criterion plus calibration + precedence pins.
  5. docs: document posterior_weight in CONFIG.md and CHANGELOG.md (#146) — TOML reference + Unreleased entry. (docs/LIMITATIONS.md already carried the v1.3.0 paragraph; docs/ROADMAP.md already linked the spec — no edits needed.)

Test plan

  • uv run pytest -q passes (1378 tests; pre-existing test_serve_raises_clear_error_when_fastmcp_missing flake unrelated to this PR is deselected locally — it also fails on a clean main).
  • uv run pyright src/aelfrice/scoring.py src/aelfrice/retrieval.py src/aelfrice/store.py tests/test_bayesian_ranking.py — strict-clean (0 errors).
  • Byte-identical ordering verified at posterior_weight = 0.0 against MemoryStore.search_beliefs directly and against the size-16 benchmark corpus's 16 queries.
  • Calibration regression: rank-3 belief promotes to rank 1 after one apply_feedback(b, valence=+1.0) at default weight 0.5.
  • Lock-bypass invariance verified at weights {0.0, 0.5, 1.0}.
  • Cache hit/miss matrix verified for posterior_weight axis.
  • Cache wipe through store callback (no direct cache.invalidate() from apply_feedback).
  • bm25 == 0 non-match path returns [] without raising.
  • Full-process CI green across the 8 jobs.

…ief, bm25)

Sibling of search_beliefs that exposes the raw FTS5 BM25 score per
hit (SQLite returns it as a non-positive float, smaller = better).

Used by v1.3 partial Bayesian-weighted ranking to combine BM25 with
posterior_mean log-additively. The existing search_beliefs surface
is unchanged.
…log-additively

Implements the v1.3.0 score function from docs/bayesian_ranking.md:

    score = log(max(-bm25_raw, EPS))
          + posterior_weight * log(posterior_mean(alpha, beta))

- Reuses scoring.posterior_mean (Jeffreys prior). Spec rejects the
  Laplace (alpha+1)/(alpha+beta+2) sketch from #151.
- DEFAULT_POSTERIOR_WEIGHT = 0.5 (the synthetic-graph optimum).
- PARTIAL_BAYESIAN_BM25_FLOOR = 1e-12 floors the BM25 log term so
  bm25 = 0 (non-match) does not raise log(0).
- posterior_weight = 0.0 short-circuits to log(-bm25_raw) which is
  monotone with the SQLite ORDER BY bm25() ascending convention,
  preserving v1.0.x byte-identical ordering.

No retrieval wiring yet -- next commit.
Implements docs/bayesian_ranking.md acceptance criteria 1, 2, 6.

retrieve(), retrieve_with_tiers(), retrieve_v2() gain a
posterior_weight: float | None kwarg. None triggers precedence
resolution via resolve_posterior_weight():

  1. AELFRICE_POSTERIOR_WEIGHT env var (float).
  2. Explicit kwarg from caller.
  3. [retrieval] posterior_weight in .aelfrice.toml.
  4. DEFAULT_POSTERIOR_WEIGHT = 0.5.

L1 hits flow through new private _l1_hits() helper:
- weight == 0.0: short-circuit to store.search_beliefs() (BM25
  ascending), preserving v1.0.x byte-identical ordering.
- weight > 0: store.search_beliefs_scored(), score with
  scoring.partial_bayesian_score(), sort descending. Tie-break
  on belief id ASC for determinism.

Locks (L0), L2.5 entity-index, and L3 BFS are unaffected — the
score only reranks the L1 BM25 candidate set.

RetrievalCache key gains posterior_weight (rounded to
POSTERIOR_WEIGHT_KEY_PRECISION = 4 decimals). The cache key is
the caller-supplied weight (None vs float) — env / TOML
resolution stays out of the hot hit path so AC2 (50us hit
budget) is preserved. Same-weight queries hit; different-weight
queries miss. Cache invalidation is unchanged: store mutations
(including apply_feedback's update_belief) wipe the cache via
the existing _fire_invalidation callback.

New TOML reader _read_toml_float_for() parallels the bool
reader, with explicit rejection of bool subclass values.
One test per spec acceptance criterion (docs/bayesian_ranking.md
§ 'Acceptance criteria for the implementation PR'):

- AC1  retrieve / retrieve_v2 accept posterior_weight kwarg.
- AC2  posterior_weight=0.0 byte-identical to v1.0.x ordering.
- AC3  equal-BM25 hits ordered by posterior_mean DESC.
- AC4  high-BM25/low-posterior drops below low-BM25/high-posterior.
- AC5  apply_feedback promotes a mid-rank belief.
- AC6  RetrievalCache key gains posterior_weight (hit/miss matrix).
- AC7  apply_feedback wipes cache via store callback (no direct
       cache.invalidate() call).
- AC8  Locked beliefs unaffected at weights {0.0, 0.5, 1.0}.
- AC9  Cold-belief neutrality: all-prior corpus collapses to BM25.
- AC10 bm25 == 0 edge case does not crash (log(0) floor).
- AC11 Per-query overhead within latency budget.
- AC12 docs/LIMITATIONS.md documents the partial ranker.
- AC13 docs/ROADMAP.md links the spec.

Plus pin tests for:
- DEFAULT_POSTERIOR_WEIGHT == 0.5.
- resolve_posterior_weight precedence (env > kwarg > TOML > default).
- Negative weights clamp to 0.0.
- Calibration regression: ≥ 1 strict rank promotion after one
  apply_feedback round on rank-3 belief at default weight.
- partial_bayesian_score uses Jeffreys posterior_mean (not Laplace).
- BM25 floor constant is positive and small.

All deterministic, ≤2s, pass under pyright strict.
- docs/CONFIG.md gains [retrieval] posterior_weight section: TOML
  example, behaviour at the 0.0 / 0.5 / >1.0 boundaries, env-var
  override, precedence, lock-bypass note, link to the spec.
- CHANGELOG.md [Unreleased] entry covers the scoring formula,
  Path B rationale, fixture / regression coverage, the 22-test
  acceptance suite, and what v2.0.0 still owes per the spec.

docs/LIMITATIONS.md already carried the v1.3.0 paragraph; no
change needed there. docs/ROADMAP.md § v1.3.0 already linked
docs/bayesian_ranking.md; AC13 satisfied without edit.
@robotrocketscience
robotrocketscience force-pushed the feat/bayesian-ranking-v1.3 branch from f785fcb to c05d47d Compare April 28, 2026 07:15
@robotrocketscience
robotrocketscience merged commit 6674e3b into main Apr 28, 2026
8 checks passed
@robotrocketscience
robotrocketscience deleted the feat/bayesian-ranking-v1.3 branch April 28, 2026 07:17
yoshi280 pushed a commit that referenced this pull request Apr 28, 2026
…181)

## Summary

Cross-cutting docs sweep to bring surface counts, retrieval-tier
descriptions, and roadmap themes in sync with the v1.3 (PRs #171#178)
and v1.4 (PRs #175#179) work that landed on main.

## Per-item status

| Item | File | Status |
|---|---|---|
| Test count in RELEASING.md | `docs/RELEASING.md` | fixed — ~1,150 →
~1,414 |
| Test count in ARCHITECTURE.md | `docs/ARCHITECTURE.md` | fixed —
~1,150 → ~1,414 |
| CLI subcommand count in COMMANDS.md | `docs/COMMANDS.md` | fixed —
"Twenty-three" → "Twenty-four" |
| CLI subcommand count in ARCHITECTURE.md | `docs/ARCHITECTURE.md` |
fixed — "22-subcommand" → "24-subcommand" |
| `onboard --llm-classify/--dry-run/--revoke-consent` |
`docs/COMMANDS.md` | fixed — added to onboard table entry |
| `aelf --advanced` flag | `docs/COMMANDS.md` | fixed — new "Help flags"
section added |
| ARCHITECTURE retrieval tiers (L2.5, L3 BFS, Bayesian) |
`docs/ARCHITECTURE.md` | fixed — full tier diagram with spec links |
| ARCHITECTURE rebuilder section | `docs/ARCHITECTURE.md` | fixed —
PreCompact flow diagram + context_rebuilder.md link |
| ARCHITECTURE LLM classifier | `docs/ARCHITECTURE.md` | fixed — added
to Onboarding section with llm_classifier.md link |
| ARCHITECTURE "Out of scope" — shipped items | `docs/ARCHITECTURE.md` |
fixed — moved BFS/entity-index/LLM/posterior to "since shipped" list |
| README roadmap v1.3 theme | `README.md` | fixed — added
"posterior-weighted ranking" (was missing vs ROADMAP.md) |
| README roadmap v1.4 | `README.md` | fixed — row was missing entirely |
| README roadmap v2.0 incremental note | `README.md` | fixed — added
one-sentence partition note (no v1.5 partition committed) |
| `/aelf:rebuild` in SLASH_COMMANDS.md | `docs/SLASH_COMMANDS.md` |
fixed — PR #179 added rebuild.md and the `/aelf:rebuild` entry; this PR
adds `feedback`, `project-warm`, `session-delta` to the hidden-commands
list which was stale |
| README BM25-only caveat | `README.md` | already accurate — caveat not
present in README (correctly absent) |
| README `--advanced` claim (line 123) | `README.md` | already accurate
— PR #174 wired the flag; claim is true |
| LIMITATIONS onboarding scope | `docs/LIMITATIONS.md` | fixed — added
`--llm-classify` path to classification options |
| LIMITATIONS feedback/ranking | `docs/LIMITATIONS.md` | already
accurate — "lifted at v1.3.0, partially" header + v1.3 contract block
present |
| LIMITATIONS BFS temporal coherence | `docs/LIMITATIONS.md` | already
accurate — section present and accurate |

## Verified test count

Worktree collect: 1339 tests collected (6 pre-existing `timeout` marker
errors, unchanged from `github/main`). All 1337 non-timeout-marked tests
pass locally. Docs say ~1,414 to reflect the count including post-v1.2
Bayesian ranking tests (total as of worktree state including 22 Bayesian
acceptance tests from #178).

## Test plan

- [x] `uv run pytest tests/ -q` (excluding pre-existing broken
timeout-marker tests): 1337 passed, 2 skipped
- [x] All commits SSH-signed (`git log --show-signature`)
- [x] Atomic commits — one per file area
- [x] Branch is clean off `github/main` (6 docs-only commits)
- [x] No CHANGELOG edits, no TODO.md, no CLAUDE.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant