robotrocketscience · robotrocketscience · Apr 28, 2026 · Apr 28, 2026 · Apr 28, 2026 · Apr 28, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -14,6 +14,8 @@ installable release; see the roadmap in [README.md](README.md).
 
 - **PreCompact hook + rebuild logic, augment mode** ([#139](https://github.com/robotrocketscience/aelfrice/issues/139), [docs/context_rebuilder.md](docs/context_rebuilder.md)). v1.4.0 milestone: replaces the v1.2.0a0 alpha's per-token union retrieval workaround with the v1.3 `retrieve()` codepath (L0 + L1 + L2.5 in one call). New `aelfrice.context_rebuilder.rebuild_v14()` pure function packs L0 locked beliefs first (full, never trimmed), then session-scoped beliefs whose `session_id` matches the latest transcript turn's session, then the L2.5/L1 tail from `retrieve()` — within a configurable token budget. Query string is built from entity + triple extraction over the recent-turn window (no LLM). `aelfrice.context_rebuilder.main()` is the new module-level Claude Code PreCompact hook entry point; `aelfrice.hook.pre_compact()` continues to dispatch through the same logic and now wraps output in the harness's `hookSpecificOutput.additionalContext` JSON envelope. New `[rebuilder] turn_window_n` and `[rebuilder] token_budget` keys in `.aelfrice.toml` (defaults 50 and 4000); CLI flags on `aelf rebuild --n` / `--budget` override per-call. `aelf rebuild` now drives the same `rebuild_v14()` codepath the hook uses. `aelf setup --rebuilder` (already shipped at v1.2.0a0) installs the PreCompact hook idempotently. Augment-mode only: both the harness's compaction summary and the rebuild block land in the new context. Suppress mode is parked for v2.x. Empty-transcript / missing-store edge cases exit 0 with no `additionalContext` written. Reproducible: same transcript tail + same store state → byte-identical envelope. Latency budget: median ≤ 200 ms on a 10k-belief store; measured ~2 ms on a workstation. 15 new deterministic tests in `tests/test_context_rebuilder_hook.py` cover ordering, edge cases, reproducibility, latency, the `[rebuilder]` config parser, the JSON envelope shape, and session-scoping invariants.
 
+- **Partial Bayesian-weighted ranking (v1.3.0)** ([#146](https://github.com/robotrocketscience/aelfrice/issues/146), [docs/bayesian_ranking.md](docs/bayesian_ranking.md)). L1 BM25 ranking now consumes the Beta-Bernoulli posterior log-additively per the spec's adopted Path B contract: `score = log(-bm25_raw) + posterior_weight * log(posterior_mean(α, β))`. `posterior_weight` defaults to `0.5` (the synthetic-graph optimum from the v1.3 calibration); `0.0` reproduces v1.0.x BM25-only ordering byte-for-byte (regression-tested). Locked beliefs (L0) bypass scoring entirely; L2.5 entity-index hits and L3 BFS expansions are unaffected — the weight only reranks the L1 candidate set. New `scoring.partial_bayesian_score(bm25_raw, alpha, beta, posterior_weight)` reuses `scoring.posterior_mean` (Jeffreys prior `α / (α+β)`); the Laplace `(α+1) / (α+β+2)` form sketched in #151 is explicitly rejected at this layer per spec rationale. New `MemoryStore.search_beliefs_scored(query, limit) -> list[tuple[Belief, float]]` exposes the FTS5 BM25 score; `MemoryStore.search_beliefs` is unchanged. `retrieve()`, `retrieve_with_tiers()`, and `retrieve_v2()` gain a `posterior_weight: float | None` kwarg. New `aelfrice.retrieval.resolve_posterior_weight()` resolves precedence env > kwarg > TOML > default; `AELFRICE_POSTERIOR_WEIGHT=<float>` env override and `[retrieval] posterior_weight = <float>` in `.aelfrice.toml`. Negative values clamp to `0.0`. `bm25 == 0` (FTS5 non-match) is floored at `PARTIAL_BAYESIAN_BM25_FLOOR = 1e-12` so `log(0)` cannot raise. `RetrievalCache` key tuple gains `posterior_weight` (rounded to four decimals via `POSTERIOR_WEIGHT_KEY_PRECISION`) so two callers passing different weights against the same store do not collide; cache invalidation is unchanged — `apply_feedback`'s `store.update_belief()` already triggers `_fire_invalidation()` and wipes the cache, no new hook in `apply_feedback`. 22 deterministic acceptance tests in `tests/test_bayesian_ranking.py` cover the 14-criterion spec (byte-identical v1.0.x at weight 0.0; equal-BM25 reranked by posterior DESC; high-BM25/low-posterior dethroned by low-BM25/high-posterior; one `apply_feedback(+1)` round promotes a rank-3 belief to ≤ 2 at default weight; lock bypass invariant across weights; cold-belief neutrality at all-prior corpus; cache hit/miss matrix; cache wiped through store callback without direct `cache.invalidate()`; bm25=0 edge case finite). Full feedback-into-ranking eval (10-round MRR uplift, ECE calibration, BM25F + heat-kernel composition, real-feedback retest) lands at v2.0.0.
+
 ### Fixed
 
 - **`project-warm`: sentinel debounce keyed off git-common-dir, not worktree path** ([#161](https://github.com/robotrocketscience/aelfrice/issues/161)). Previously `_project_id` was derived from `git rev-parse --show-toplevel`, giving each worktree of the same repo a distinct sentinel under `~/.aelfrice/projects/<id>/.last_warm`. Two worktrees of one repo share a single DB (via `git-common-dir`), so they should share one sentinel. `resolve_project_root` now calls `git rev-parse --path-format=absolute --show-toplevel --git-common-dir` in a single subprocess and keys `ProjectRef.id` off the git-common-dir while keeping `ProjectRef.root` as the worktree working directory (for `os.chdir` in `_warm_store`). New test `test_resolve_project_root_worktrees_share_id` verifies that two worktrees of one repo produce identical `ProjectRef.id` values.

diff --git a/README.md b/README.md
@@ -134,8 +134,9 @@ The same operations are also available as MCP tools and `/aelf:*` slash commands
 | v1.1.0 | shipped | per-project DBs (`.git/aelfrice/`), `aelf migrate`, `edges`→`threads` rename, `aelf health` rewrite |
 | v1.2.0 | shipped | auto-capture pipeline (transcript-ingest, commit-ingest, SessionStart), `agent_inferred → user_validated` promotion, triple extractor, `--batch` JSONL ingest, CLI consolidation, `INEDIBLE` per-file opt-out |
 | v1.2.x | planned | search-tool `PreToolUse` hook — memory-first context on Grep/Glob |
-| v1.3 | planned | retrieval wave — entity index + BFS multi-hop + LLM classification |
-| v2.0 | planned | feature parity with the original research line + benchmark reproducibility |
+| v1.3 | planned | retrieval wave — entity index + BFS multi-hop + LLM classification + posterior-weighted ranking |
+| v1.4 | planned | context rebuilder — PreCompact retrieval-curated continuation |
+| v2.0 | planned | feature parity with the original research line + benchmark reproducibility. v2.0's component issues (#148–#154) will land incrementally across v1.5+ minor versions; final v2.0 tag is the reproducibility cut. |
 
 Per-version detail: [docs/ROADMAP.md](docs/ROADMAP.md). Open issues: [docs/LIMITATIONS.md](docs/LIMITATIONS.md).
 

diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
@@ -6,7 +6,7 @@ How aelfrice fits together. Maps directly to source under `src/aelfrice/`.
 
 1. **Determinism end to end.** Every retrieval result is bit-identical given the same write log and the same code. Every result traces to named beliefs and named rules. See [PHILOSOPHY § Determinism is the property](PHILOSOPHY.md#determinism-is-the-property).
 2. **Stdlib + SQLite only.** No vector DB, no embeddings, no LLM in the hot path. The `[mcp]` extra (`fastmcp`) is the only optional runtime dep.
-3. **Bayesian, not vibes.** Confidence is `α / (α + β)`. Every update has a closed-form rule. (At v1.0–v1.2 the score does not yet drive ranking — see [LIMITATIONS](LIMITATIONS.md).)
+3. **Bayesian, not vibes.** Confidence is `α / (α + β)`. Every update has a closed-form rule. At v1.3.0+ the posterior is combined log-additively with BM25 on the L1 tier — see [LIMITATIONS](LIMITATIONS.md) for what the partial ranking does and doesn't cover.
 4. **`apply_feedback` is the central endpoint.** One writer of `(α, β)`. One audit row per successful update.
 5. **Locks are user-asserted ground truth.** A user-locked belief short-circuits decay. Contradicting positive feedback accumulates `demotion_pressure`; ≥5 ⇒ auto-demote.
 
@@ -29,7 +29,7 @@ Imports are one-directional — modules lower in the table import from higher.
 | `models.py` | `Belief`, `Edge`, `FeedbackEvent`, `OnboardSession` dataclasses; type / lock / origin constants. No I/O. |
 | `scoring.py` | `posterior_mean`, `decay`, `relevance_combiner`. Type half-lives. Lock-floor short-circuit. Decay target: Jeffreys `(0.5, 0.5)`. |
 | `store.py` | SQLite WAL + FTS5 + CRUD. `propagate_valence` BFS with broker-confidence attenuation. |
-| `retrieval.py` | `retrieve(store, query, token_budget=2000)` — L0 locked + L1 FTS5 BM25. L0 never trimmed. |
+| `retrieval.py` | `retrieve(store, query, token_budget=2000)` — L0 locked + L2.5 entity-index (v1.3+) + L3 BFS multi-hop (v1.3+, default-off) + L1 FTS5 BM25 with Bayesian log-additive reranking (v1.3+). L0 never trimmed. |
 | `feedback.py` | `apply_feedback(store, belief_id, valence, source)` — only Bayesian-update path. Writes `feedback_history`. Drives demotion-pressure + auto-demote. |
 | `contradiction.py` | `resolve_contradiction` — picks a winner per precedence, inserts `SUPERSEDES`, writes audit row. Backs `aelf resolve`. |
 | `correction.py` | No-LLM heuristic correction detector. |
@@ -47,7 +47,7 @@ Imports are one-directional — modules lower in the table import from higher.
 | `triple_extractor.py` | Pure-regex `(subject, relation, object)` extraction over six relation families. Used by commit-ingest and transcript-ingest. |
 | `context_rebuilder.py` | PreCompact alpha that surfaces aelfrice retrieval before Claude Code summarises. |
 | `benchmark.py` | Deterministic 16-belief × 16-query synthetic harness. Frozen `BenchmarkReport`. |
-| `cli.py` | argparse 22-subcommand CLI. Entry: `aelf`. |
+| `cli.py` | argparse 24-subcommand CLI. Entry: `aelf`. |
 | `mcp_server.py` | FastMCP server, 9 tools. `[mcp]` optional extra. |
 | `setup.py` | Idempotent install/uninstall of all hooks + statusline. Atomic write via tempfile + `os.replace`. |
 | `hook.py` | `aelfrice.hook:main` — process Claude Code spawns on each prompt. Reads stdin, calls `retrieve()`, emits `<aelfrice-memory>` on stdout. Non-blocking. Entry: `aelf-hook`. |
@@ -89,18 +89,24 @@ Walk is 1-hop only. Multi-hop pressure is deferred.
 ## Retrieval
 
 ```
-L0: store.list_locked()         always loaded; never trimmed
+L0: store.list_locked()              always loaded; never trimmed
         ↓
-L1: FTS5 BM25 keyword search    limit l1_limit, query escaped
+L2.5: entity-index lookup (v1.3+)    NER-extracted entities → exact + stem match;
+        ↓                             default-on; disable via [retrieval] entity_index_enabled = false
+L3: BFS multi-hop expansion (v1.3+)  edge-weighted graph walk from L0+L2.5 seeds;
+        ↓                             default-OFF; enable via [retrieval] bfs_enabled = true
+L1: FTS5 BM25 keyword search         limit l1_limit, query escaped;
+        ↓                             v1.3+: score = log(bm25) + 0.5*log(posterior_mean)
+Dedupe L1+L2.5+L3 against L0 ids
         ↓
-Dedupe L1 against L0 ids
-        ↓
-Trim L1 from tail until sum(estimated_tokens) ≤ token_budget
+Trim from tail until sum(estimated_tokens) ≤ token_budget
 ```
 
 Token estimate: `(len(content) + 3) // 4`. Empty query: L0 only. L0 always wins overflow.
 
-The v1.3.0 retrieval wave inserts an L2.5 entity-index tier between L0 and L1 — spec lives at [entity_index.md](entity_index.md).
+Spec docs: [entity_index.md](entity_index.md) (L2.5), [bfs_multihop.md](bfs_multihop.md) (L3), [bayesian_ranking.md](bayesian_ranking.md) (L1 Bayesian reranking).
+
+**BFS temporal-coherence caveat:** L3 resolves each hop to the globally latest serial of its target belief. For recall queries this is correct. For audit queries (what did the agent believe at decision-time?) a post-seed supersession can appear mid-chain. The temporal-coherence fix is targeted at v2.0.0 — see [LIMITATIONS § BFS multi-hop temporal coherence](LIMITATIONS.md#bfs-multi-hop-temporal-coherence).
 
 ## Onboarding
 
@@ -112,6 +118,8 @@ The v1.3.0 retrieval wave inserts an L2.5 entity-index tier between L0 and L1
 
 Classification via priors + regex fallback. Idempotent on `content_hash`.
 
+**LLM-Haiku onboard classifier (v1.3+, default-OFF):** `aelf onboard --llm-classify` routes each candidate through Claude Haiku instead of the regex path. Four consent gates enforce the privacy boundary: flag presence, `ANTHROPIC_API_KEY` present, stored sentinel, interactive prompt. `--dry-run` previews candidates without calling the API. Spec: [llm_classifier.md](llm_classifier.md). This is the only path in aelfrice that transmits user content outbound — see [PRIVACY § Optional outbound calls](PRIVACY.md#optional-outbound-calls).
+
 ## Claude Code hook
 
 ```
@@ -145,6 +153,27 @@ observation produced by a HOME-side hook (tracked separately). See
 [hook_activity_schema](hook_activity_schema.md) for the field schema
 and the consumer-side dedupe-by-fingerprint warning.
 
+## PreCompact rebuilder (v1.4)
+
+When Claude Code approaches its context limit it fires `PreCompact`. The `aelf-pre-compact-hook` intercepts this event and injects a curated retrieval block before the harness summarises:
+
+```
+PreCompact fires
+      ↓
+aelf-pre-compact-hook reads the last N turns from turns.jsonl
+      ↓
+rebuild_v14(recent_turns, store, token_budget)
+      → L0 locked beliefs (always first)
+      → session-scoped beliefs matching recent content
+      → BM25+posterior hits against the session tail
+      packed to token_budget (default: [rebuilder].token_budget in .aelfrice.toml)
+      ↓
+emitted as additionalContext — both the aelfrice block
+and the harness's own summary land in the new context (augment mode)
+```
+
+`aelf rebuild [--transcript PATH] [--n N] [--budget N]` runs the same codepath manually (prints block to stdout). Install via `aelf setup --rebuilder`. Spec: [context_rebuilder.md](context_rebuilder.md). Eval fixture policy: [eval_fixture_policy.md](eval_fixture_policy.md).
+
 ## Tests
 
 | Layer | Marker | Coverage |
@@ -153,15 +182,18 @@ and the consumer-side dedupe-by-fingerprint warning.
 | Property | default | Pre-registered invariants: Bayesian inertia, decay-required, lock-floor sharpness, token-budget invariant, broker-attenuation. |
 | Regression | `@pytest.mark.regression` | Cross-module scenarios: retrieval round-trip, feedback loop, onboarding, setup→hook→unsetup, `aelf bench` end-to-end. |
 
-`uv run pytest` (~1,150 tests at v1.2, ~15s on Apple Silicon).
+`uv run pytest` (~1,414 tests at v1.3/v1.4, ~15s on Apple Silicon).
 
 ## Out of scope through v1.x
 
 These land at v2.0 with evidence (a benchmark, an experiment, a clear case where the existing operations don't suffice):
 
-- Posterior-aware retrieval ranking (gated on the v1.3 retrieval wave)
 - HRR / sentence-transformer embeddings
-- BFS multi-hop graph retrieval
-- Entity index / NER
-- LLM in the hot path
 - Cross-project knowledge federation
+- Full posterior-driven ranking eval (10-round MRR uplift, ECE calibration, BM25F + heat-kernel composition — v2.0.0; the partial Bayesian reranking shipped at v1.3.0)
+
+The following were previously listed here and have since shipped:
+- Posterior-aware retrieval ranking → **shipped v1.3.0** (partial; [bayesian_ranking.md](bayesian_ranking.md))
+- BFS multi-hop graph retrieval → **shipped v1.3.0** ([bfs_multihop.md](bfs_multihop.md))
+- Entity index / NER → **shipped v1.3.0** ([entity_index.md](entity_index.md))
+- LLM in the hot path (optional onboard classifier) → **shipped v1.3.0** ([llm_classifier.md](llm_classifier.md))
diff --git a/docs/COMMANDS.md b/docs/COMMANDS.md
@@ -1,6 +1,6 @@
 # Commands
 
-Twenty-three CLI subcommands. The retrieval/feedback ones are also exposed as MCP tools (see [MCP](MCP.md)) and slash commands (see [SLASH_COMMANDS](SLASH_COMMANDS.md)). Lifecycle commands (`setup`, `doctor`, `migrate`, `upgrade`, `uninstall`, etc.) are CLI-only.
+Twenty-four CLI subcommands. The retrieval/feedback ones are also exposed as MCP tools (see [MCP](MCP.md)) and slash commands (see [SLASH_COMMANDS](SLASH_COMMANDS.md)). Lifecycle commands (`setup`, `doctor`, `migrate`, `upgrade`, `uninstall`, etc.) are CLI-only.
 
 ```
 aelf <subcommand> [args] [options]
@@ -14,7 +14,7 @@ DB resolves from `$AELFRICE_DB`, then `<git-common-dir>/aelfrice/memory.db` when
 
 | Command | What it does |
 |---|---|
-| `onboard <path>` | Walk filesystem, git log, Python AST. Classify candidates, insert non-duplicates. Tunable via `.aelfrice.toml` — see [CONFIG](CONFIG.md). |
+| `onboard <path>` | Walk filesystem, git log, Python AST. Classify candidates, insert non-duplicates. Tunable via `.aelfrice.toml` — see [CONFIG](CONFIG.md). Optional flags (v1.3+): `--llm-classify` (route through Haiku classifier; default-off, requires `ANTHROPIC_API_KEY`), `--dry-run` (preview candidates without inserting; requires `--llm-classify`), `--revoke-consent` (remove the stored consent sentinel and exit). |
 | `search <query> [--budget N]` | L0 locked + L2.5 entity-index (v1.3+) + L1 FTS5 BM25, token-budgeted (default 2,400 at v1.3+, 2,000 prior). L2.5 default-on; disable via `[retrieval] entity_index_enabled = false` in `.aelfrice.toml` or `AELFRICE_ENTITY_INDEX=0` in the env. Distinguishes "store empty" from "no match". |
 | `lock <statement>` | Insert at `(α, β) = (9.0, 0.5)` with `lock_level=user`. Idempotent — re-lock upgrades existing. |
 | `locked [--pressured]` | List locks. With `--pressured`, only those with `demotion_pressure > 0`. |
@@ -49,6 +49,10 @@ DB resolves from `$AELFRICE_DB`, then `<git-common-dir>/aelfrice/memory.db` when
 | `project-warm <path> [--debounce N]` | CwdChanged hook entry point. Resolves `<path>` to a project root (git work-tree or `~/.aelfrice/projects/<id>/`-provisioned ancestor), pre-loads the SQLite + OS page cache, and writes a sentinel under `~/.aelfrice/projects/<id>/.last_warm`. Silent no-op for unknown paths, denied paths (default deny: `/tmp/**`, `/var/folders/**`, `~/Downloads/**`, `~/Desktop/**` — override via `~/.aelfrice/config.json` `project_warm.deny_globs`), and any call inside the 60-second debounce window. Always exits 0; never writes to stdout. |
 | `session-delta [--id ID] [--telemetry-path PATH]` | **Advanced/hidden.** SessionEnd hook entry point. Computes per-session deltas (beliefs created, corrections detected, feedback given, velocity) from beliefs tagged with `--id` in the active store, combines with a current store snapshot (beliefs/graph blocks) and rolling-window rollups from the existing `telemetry.jsonl`, and appends one v=1 JSON row to `PATH` (default `~/.aelfrice/telemetry.jsonl`). Missing or empty `--id` is a silent no-op (stderr warning, exit 0). Idle sessions with zero beliefs still emit a row so `len(telemetry.jsonl)` equals session count. Not shown in `aelf --help`. |
 
+## Help flags
+
+`aelf --help` shows the everyday surface (visible subcommands). `aelf --help --advanced` (or `aelf --advanced`) shows the full surface including hidden subcommands (`bench`, `feedback`, `health`, `migrate`, `project-warm`, `rebuild`, `regime`, `session-delta`, `stats`, `statusline`, `unsetup`). The `--advanced` flag was wired in v1.4 (PR #174).
+
 ## Output and exit codes
 
 - Human-readable on stdout. Errors on stderr.