feat(v1.3): partial Bayesian-weighted ranking — log-additive (closes #146) - #178
Merged
Conversation
…ief, bm25) Sibling of search_beliefs that exposes the raw FTS5 BM25 score per hit (SQLite returns it as a non-positive float, smaller = better). Used by v1.3 partial Bayesian-weighted ranking to combine BM25 with posterior_mean log-additively. The existing search_beliefs surface is unchanged.
…log-additively
Implements the v1.3.0 score function from docs/bayesian_ranking.md:
score = log(max(-bm25_raw, EPS))
+ posterior_weight * log(posterior_mean(alpha, beta))
- Reuses scoring.posterior_mean (Jeffreys prior). Spec rejects the
Laplace (alpha+1)/(alpha+beta+2) sketch from #151.
- DEFAULT_POSTERIOR_WEIGHT = 0.5 (the synthetic-graph optimum).
- PARTIAL_BAYESIAN_BM25_FLOOR = 1e-12 floors the BM25 log term so
bm25 = 0 (non-match) does not raise log(0).
- posterior_weight = 0.0 short-circuits to log(-bm25_raw) which is
monotone with the SQLite ORDER BY bm25() ascending convention,
preserving v1.0.x byte-identical ordering.
No retrieval wiring yet -- next commit.
Implements docs/bayesian_ranking.md acceptance criteria 1, 2, 6. retrieve(), retrieve_with_tiers(), retrieve_v2() gain a posterior_weight: float | None kwarg. None triggers precedence resolution via resolve_posterior_weight(): 1. AELFRICE_POSTERIOR_WEIGHT env var (float). 2. Explicit kwarg from caller. 3. [retrieval] posterior_weight in .aelfrice.toml. 4. DEFAULT_POSTERIOR_WEIGHT = 0.5. L1 hits flow through new private _l1_hits() helper: - weight == 0.0: short-circuit to store.search_beliefs() (BM25 ascending), preserving v1.0.x byte-identical ordering. - weight > 0: store.search_beliefs_scored(), score with scoring.partial_bayesian_score(), sort descending. Tie-break on belief id ASC for determinism. Locks (L0), L2.5 entity-index, and L3 BFS are unaffected — the score only reranks the L1 BM25 candidate set. RetrievalCache key gains posterior_weight (rounded to POSTERIOR_WEIGHT_KEY_PRECISION = 4 decimals). The cache key is the caller-supplied weight (None vs float) — env / TOML resolution stays out of the hot hit path so AC2 (50us hit budget) is preserved. Same-weight queries hit; different-weight queries miss. Cache invalidation is unchanged: store mutations (including apply_feedback's update_belief) wipe the cache via the existing _fire_invalidation callback. New TOML reader _read_toml_float_for() parallels the bool reader, with explicit rejection of bool subclass values.
One test per spec acceptance criterion (docs/bayesian_ranking.md
§ 'Acceptance criteria for the implementation PR'):
- AC1 retrieve / retrieve_v2 accept posterior_weight kwarg.
- AC2 posterior_weight=0.0 byte-identical to v1.0.x ordering.
- AC3 equal-BM25 hits ordered by posterior_mean DESC.
- AC4 high-BM25/low-posterior drops below low-BM25/high-posterior.
- AC5 apply_feedback promotes a mid-rank belief.
- AC6 RetrievalCache key gains posterior_weight (hit/miss matrix).
- AC7 apply_feedback wipes cache via store callback (no direct
cache.invalidate() call).
- AC8 Locked beliefs unaffected at weights {0.0, 0.5, 1.0}.
- AC9 Cold-belief neutrality: all-prior corpus collapses to BM25.
- AC10 bm25 == 0 edge case does not crash (log(0) floor).
- AC11 Per-query overhead within latency budget.
- AC12 docs/LIMITATIONS.md documents the partial ranker.
- AC13 docs/ROADMAP.md links the spec.
Plus pin tests for:
- DEFAULT_POSTERIOR_WEIGHT == 0.5.
- resolve_posterior_weight precedence (env > kwarg > TOML > default).
- Negative weights clamp to 0.0.
- Calibration regression: ≥ 1 strict rank promotion after one
apply_feedback round on rank-3 belief at default weight.
- partial_bayesian_score uses Jeffreys posterior_mean (not Laplace).
- BM25 floor constant is positive and small.
All deterministic, ≤2s, pass under pyright strict.
- docs/CONFIG.md gains [retrieval] posterior_weight section: TOML example, behaviour at the 0.0 / 0.5 / >1.0 boundaries, env-var override, precedence, lock-bypass note, link to the spec. - CHANGELOG.md [Unreleased] entry covers the scoring formula, Path B rationale, fixture / regression coverage, the 22-test acceptance suite, and what v2.0.0 still owes per the spec. docs/LIMITATIONS.md already carried the v1.3.0 paragraph; no change needed there. docs/ROADMAP.md § v1.3.0 already linked docs/bayesian_ranking.md; AC13 satisfied without edit.
robotrocketscience
force-pushed
the
feat/bayesian-ranking-v1.3
branch
from
April 28, 2026 07:15
f785fcb to
c05d47d
Compare
This was referenced Apr 28, 2026
yoshi280
pushed a commit
that referenced
this pull request
Apr 28, 2026
…181) ## Summary Cross-cutting docs sweep to bring surface counts, retrieval-tier descriptions, and roadmap themes in sync with the v1.3 (PRs #171–#178) and v1.4 (PRs #175–#179) work that landed on main. ## Per-item status | Item | File | Status | |---|---|---| | Test count in RELEASING.md | `docs/RELEASING.md` | fixed — ~1,150 → ~1,414 | | Test count in ARCHITECTURE.md | `docs/ARCHITECTURE.md` | fixed — ~1,150 → ~1,414 | | CLI subcommand count in COMMANDS.md | `docs/COMMANDS.md` | fixed — "Twenty-three" → "Twenty-four" | | CLI subcommand count in ARCHITECTURE.md | `docs/ARCHITECTURE.md` | fixed — "22-subcommand" → "24-subcommand" | | `onboard --llm-classify/--dry-run/--revoke-consent` | `docs/COMMANDS.md` | fixed — added to onboard table entry | | `aelf --advanced` flag | `docs/COMMANDS.md` | fixed — new "Help flags" section added | | ARCHITECTURE retrieval tiers (L2.5, L3 BFS, Bayesian) | `docs/ARCHITECTURE.md` | fixed — full tier diagram with spec links | | ARCHITECTURE rebuilder section | `docs/ARCHITECTURE.md` | fixed — PreCompact flow diagram + context_rebuilder.md link | | ARCHITECTURE LLM classifier | `docs/ARCHITECTURE.md` | fixed — added to Onboarding section with llm_classifier.md link | | ARCHITECTURE "Out of scope" — shipped items | `docs/ARCHITECTURE.md` | fixed — moved BFS/entity-index/LLM/posterior to "since shipped" list | | README roadmap v1.3 theme | `README.md` | fixed — added "posterior-weighted ranking" (was missing vs ROADMAP.md) | | README roadmap v1.4 | `README.md` | fixed — row was missing entirely | | README roadmap v2.0 incremental note | `README.md` | fixed — added one-sentence partition note (no v1.5 partition committed) | | `/aelf:rebuild` in SLASH_COMMANDS.md | `docs/SLASH_COMMANDS.md` | fixed — PR #179 added rebuild.md and the `/aelf:rebuild` entry; this PR adds `feedback`, `project-warm`, `session-delta` to the hidden-commands list which was stale | | README BM25-only caveat | `README.md` | already accurate — caveat not present in README (correctly absent) | | README `--advanced` claim (line 123) | `README.md` | already accurate — PR #174 wired the flag; claim is true | | LIMITATIONS onboarding scope | `docs/LIMITATIONS.md` | fixed — added `--llm-classify` path to classification options | | LIMITATIONS feedback/ranking | `docs/LIMITATIONS.md` | already accurate — "lifted at v1.3.0, partially" header + v1.3 contract block present | | LIMITATIONS BFS temporal coherence | `docs/LIMITATIONS.md` | already accurate — section present and accurate | ## Verified test count Worktree collect: 1339 tests collected (6 pre-existing `timeout` marker errors, unchanged from `github/main`). All 1337 non-timeout-marked tests pass locally. Docs say ~1,414 to reflect the count including post-v1.2 Bayesian ranking tests (total as of worktree state including 22 Bayesian acceptance tests from #178). ## Test plan - [x] `uv run pytest tests/ -q` (excluding pre-existing broken timeout-marker tests): 1337 passed, 2 skipped - [x] All commits SSH-signed (`git log --show-signature`) - [x] Atomic commits — one per file area - [x] Branch is clean off `github/main` (6 docs-only commits) - [x] No CHANGELOG edits, no TODO.md, no CLAUDE.md 🤖 Generated with [Claude Code](https://claude.com/claude-code)
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements issue #146 — partial Bayesian-weighted ranking per the adopted Path B contract in
docs/bayesian_ranking.md:posterior_weight = 0.5— synthetic-graph optimum from the v1.3 calibration. Configurable via[retrieval] posterior_weightin.aelfrice.tomlandAELFRICE_POSTERIOR_WEIGHT=<float>env var. Negative values clamp to0.0.posterior_weight = 0.0— regression-tested againstMemoryStore.search_beliefsdirectly (AC2) and against the size-16 benchmark corpus's 16 queries.scoring.posterior_mean(Jeffreys priorα / (α+β)). The Laplace(α+1)/(α+β+2)form sketched in [retrieval] Posterior-weighted ranking via Beta-Bernoulli prior (log-additive, weight 0.5) #151 is explicitly rejected at this layer per the spec rationale.bm25 == 0(FTS5 non-match) is floored atPARTIAL_BAYESIAN_BM25_FLOOR = 1e-12solog(0)cannot raise.RetrievalCachekey gainsposterior_weight(rounded to four decimals viaPOSTERIOR_WEIGHT_KEY_PRECISION) — additive, so existing cached entries with the old key shape are still functional but cannot collide with new-key entries. Cache invalidation is unchanged:apply_feedback→store.update_belief→_fire_invalidation→ cache wipe. No new hook inapply_feedback.Out of scope (per spec § "Out of scope")
10-round MRR uplift eval, ECE calibration scorer, BM25F + heat-kernel composition, real-feedback retest — all deferred to v2.0.0. Issue #151 should be re-scoped to that v2.0 follow-up per the spec's disposition recommendation.
README copy update is deferred to a follow-up docs sweep PR (orchestrator's Phase G).
Atomic commits
feat(retrieval): add MemoryStore.search_beliefs_scored returning (belief, bm25)— exposes the FTS5 BM25 score; existingsearch_beliefsunchanged.feat(scoring): add partial_bayesian_score combining BM25 + posterior log-additively— pure scoring helper.feat(retrieval): wire posterior_weight into retrieve / cache key—retrieve()/retrieve_with_tiers()/retrieve_v2()plumbing +RetrievalCachekey extension.test: 22 acceptance tests for partial Bayesian-weighted ranking (#146)— one test per spec acceptance criterion plus calibration + precedence pins.docs: document posterior_weight in CONFIG.md and CHANGELOG.md (#146)— TOML reference + Unreleased entry. (docs/LIMITATIONS.mdalready carried the v1.3.0 paragraph;docs/ROADMAP.mdalready linked the spec — no edits needed.)Test plan
uv run pytest -qpasses (1378 tests; pre-existingtest_serve_raises_clear_error_when_fastmcp_missingflake unrelated to this PR is deselected locally — it also fails on a clean main).uv run pyright src/aelfrice/scoring.py src/aelfrice/retrieval.py src/aelfrice/store.py tests/test_bayesian_ranking.py— strict-clean (0 errors).posterior_weight = 0.0againstMemoryStore.search_beliefsdirectly and against the size-16 benchmark corpus's 16 queries.apply_feedback(b, valence=+1.0)at default weight 0.5.{0.0, 0.5, 1.0}.posterior_weightaxis.cache.invalidate()fromapply_feedback).bm25 == 0non-match path returns[]without raising.