feat(v1.4): rebuilder trigger modes — manual + threshold (+ dynamic parked) (closes #141) - #179
Merged
Merged
Conversation
…ief, bm25) Sibling of search_beliefs that exposes the raw FTS5 BM25 score per hit (SQLite returns it as a non-positive float, smaller = better). Used by v1.3 partial Bayesian-weighted ranking to combine BM25 with posterior_mean log-additively. The existing search_beliefs surface is unchanged.
…log-additively
Implements the v1.3.0 score function from docs/bayesian_ranking.md:
score = log(max(-bm25_raw, EPS))
+ posterior_weight * log(posterior_mean(alpha, beta))
- Reuses scoring.posterior_mean (Jeffreys prior). Spec rejects the
Laplace (alpha+1)/(alpha+beta+2) sketch from #151.
- DEFAULT_POSTERIOR_WEIGHT = 0.5 (the synthetic-graph optimum).
- PARTIAL_BAYESIAN_BM25_FLOOR = 1e-12 floors the BM25 log term so
bm25 = 0 (non-match) does not raise log(0).
- posterior_weight = 0.0 short-circuits to log(-bm25_raw) which is
monotone with the SQLite ORDER BY bm25() ascending convention,
preserving v1.0.x byte-identical ordering.
No retrieval wiring yet -- next commit.
Implements docs/bayesian_ranking.md acceptance criteria 1, 2, 6. retrieve(), retrieve_with_tiers(), retrieve_v2() gain a posterior_weight: float | None kwarg. None triggers precedence resolution via resolve_posterior_weight(): 1. AELFRICE_POSTERIOR_WEIGHT env var (float). 2. Explicit kwarg from caller. 3. [retrieval] posterior_weight in .aelfrice.toml. 4. DEFAULT_POSTERIOR_WEIGHT = 0.5. L1 hits flow through new private _l1_hits() helper: - weight == 0.0: short-circuit to store.search_beliefs() (BM25 ascending), preserving v1.0.x byte-identical ordering. - weight > 0: store.search_beliefs_scored(), score with scoring.partial_bayesian_score(), sort descending. Tie-break on belief id ASC for determinism. Locks (L0), L2.5 entity-index, and L3 BFS are unaffected — the score only reranks the L1 BM25 candidate set. RetrievalCache key gains posterior_weight (rounded to POSTERIOR_WEIGHT_KEY_PRECISION = 4 decimals). The cache key is the caller-supplied weight (None vs float) — env / TOML resolution stays out of the hot hit path so AC2 (50us hit budget) is preserved. Same-weight queries hit; different-weight queries miss. Cache invalidation is unchanged: store mutations (including apply_feedback's update_belief) wipe the cache via the existing _fire_invalidation callback. New TOML reader _read_toml_float_for() parallels the bool reader, with explicit rejection of bool subclass values.
One test per spec acceptance criterion (docs/bayesian_ranking.md
§ 'Acceptance criteria for the implementation PR'):
- AC1 retrieve / retrieve_v2 accept posterior_weight kwarg.
- AC2 posterior_weight=0.0 byte-identical to v1.0.x ordering.
- AC3 equal-BM25 hits ordered by posterior_mean DESC.
- AC4 high-BM25/low-posterior drops below low-BM25/high-posterior.
- AC5 apply_feedback promotes a mid-rank belief.
- AC6 RetrievalCache key gains posterior_weight (hit/miss matrix).
- AC7 apply_feedback wipes cache via store callback (no direct
cache.invalidate() call).
- AC8 Locked beliefs unaffected at weights {0.0, 0.5, 1.0}.
- AC9 Cold-belief neutrality: all-prior corpus collapses to BM25.
- AC10 bm25 == 0 edge case does not crash (log(0) floor).
- AC11 Per-query overhead within latency budget.
- AC12 docs/LIMITATIONS.md documents the partial ranker.
- AC13 docs/ROADMAP.md links the spec.
Plus pin tests for:
- DEFAULT_POSTERIOR_WEIGHT == 0.5.
- resolve_posterior_weight precedence (env > kwarg > TOML > default).
- Negative weights clamp to 0.0.
- Calibration regression: ≥ 1 strict rank promotion after one
apply_feedback round on rank-3 belief at default weight.
- partial_bayesian_score uses Jeffreys posterior_mean (not Laplace).
- BM25 floor constant is positive and small.
All deterministic, ≤2s, pass under pyright strict.
- docs/CONFIG.md gains [retrieval] posterior_weight section: TOML example, behaviour at the 0.0 / 0.5 / >1.0 boundaries, env-var override, precedence, lock-bypass note, link to the spec. - CHANGELOG.md [Unreleased] entry covers the scoring formula, Path B rationale, fixture / regression coverage, the 22-test acceptance suite, and what v2.0.0 still owes per the spec. docs/LIMITATIONS.md already carried the v1.3.0 paragraph; no change needed there. docs/ROADMAP.md § v1.3.0 already linked docs/bayesian_ranking.md; AC13 satisfied without edit.
Adds [rebuilder] trigger_mode and threshold_fraction to RebuilderConfig, defaulting to "manual" / 0.7 at v1.4.0. The PreCompact hook now no-ops when trigger_mode='manual' so the rebuilder fires only on explicit user invocation. trigger_mode='dynamic' logs a "parked v1.5" trace and no-ops; threshold-mode behaviour is unchanged. Promotes the previously-hidden `aelf rebuild` subparser to a visible verb and ships /aelf:rebuild as the manual surface (slash file in src/aelfrice/slash_commands/rebuild.md). Matches the spec's "Manual ships first as the explicit testing surface" sequencing rule. Existing hook regression tests (test_context_rebuilder_hook.py) opt into trigger_mode='threshold' via a new _write_threshold_config() helper; the slash-command surface test moves `rebuild` from the HIDDEN_SUBCOMMANDS set into EXPECTED_COMMANDS.
Adds the "threshold" trigger mode and seeds its default threshold_fraction (0.6) from the eval-harness calibration, not from a hand-picked number. New benchmarks/context_rebuilder/calibrate.py sweeps threshold fractions 0.5/0.6/0.7/0.8/0.9 against the bundled synthetic fixture, seeds an in-memory MemoryStore with one belief per pre-clear assistant turn, calls rebuild_v14() at a compressed token_budget (200) so the retrieved-beliefs section has to choose what to surface, and scores each post-clear assistant turn by content- overlap on the <retrieved-beliefs> sub-block. Choice rule: filter to ratio <= 1.5, max efficiency (fidelity/ratio) rounded to 3dp, lowest threshold breaks ties. The committed JSON at benchmarks/context-rebuilder/calibration_v1_4_0.json is the reproducible source-of-truth; DEFAULT_THRESHOLD_FRACTION pins to the chosen value. When trigger_mode='threshold', the PreCompact hook fires as in v1.2.0a0 (the harness's own PreCompact firing is the gate; threshold_fraction documents the calibrated operating point). docs/context_rebuilder.md gains § Threshold calibration with the sweep table + choice rule + per-fixture caveat. The shipped-vs- parked acceptance table is updated. Pyright strict-clean on every touched file. Dynamic-mode investigation lands in the next commit.
…141) New benchmarks/context_rebuilder/dynamic_probe.py runs two heuristic-driven trigger candidates against the bundled synthetic fixture and emits a JSON verdict against the spec ship-gate ("dynamic mode beats threshold by >= 5% absolute fidelity at same-or-lower token cost"). Candidate 1 (rate-of-context-growth): fires at first turn whose 4-turn rolling per-turn token average exceeds 1.5x the fixture- wide median. On the synthetic fixture the rule never trips and falls back to the last turn -> token ratio 1.461 (vs threshold 1.017) fails the cost half of the gate. Candidate 2 (entity-density-delta): fires at first turn (after a 4-turn warmup) whose new-entity count drops below half the fixture median. Fires at turn 4 with fidelity 0.0824 and ratio 0.682 (cheaper than threshold's 1.017) -- but fidelity delta of +0.0147 absolute is well below the +0.05 spec gate. Verdict: park. docs/context_rebuilder.md gains § Dynamic mode (parked v1.5) with the full measurement table and park rationale. The implementation stays in the tree so a v1.5.x re-investigation can build on the same proxy + fixture without re-deriving them. The dynamic-mode hook gate (already in place from the manual-mode commit) logs a "parked v1.5" trace to stderr and no-ops when trigger_mode = "dynamic" is set in .aelfrice.toml.
New tests/test_rebuilder_triggers.py adds 16 deterministic acceptance tests for the trigger-mode behaviour: - TM1: manual is the v1.4 default; PreCompact hook no-ops in manual mode; aelf rebuild bypasses the trigger gate (manual surface always works). - TM2: threshold mode emits the envelope; threshold default matches the calibration JSON's chosen value; out-of-range threshold_fraction degrades to default; .aelfrice.toml override wins. - TM3: calibration JSON is committed and well-formed; running calibrate.py re-derives byte-for-byte (excluding the caller- chosen `fixture` path field). - TM4: dynamic is in the valid-modes set; setting it logs a "parked v1.5" trace to stderr from both pre_compact() and context_rebuilder.main(), and writes nothing to stdout. - TM5: dynamic_probe verdict is "park" on the synthetic fixture (regression-test on the parking decision; flips CI red if a future change would re-enable dynamic mode without docs). - Edge: parser rejects unknown trigger_mode strings; every valid mode round-trips through the parser. Plus two existing-test fixups required by the manual-mode-default flip: - tests/test_hook_pre_compact.py: write `[rebuilder] trigger_mode = "threshold"` in `_aelfrice_log` and the falls-back-to-Claude- transcript test so the auto-fire-path tests stay valid. - tests/test_cli_advanced_help.py: `rebuild` is no longer hidden from --help (it became the user-facing manual-trigger surface). uv run pytest -q stays green: 1430 passed, 2 skipped. Pyright strict-clean on test_rebuilder_triggers.py.
CHANGELOG.md [Unreleased] gains an Added entry covering the manual + threshold ship + dynamic-parking decision, with pointers to the calibration JSON, the dynamic_probe ship-gate result, and the regression tests. ROADMAP.md § v1.4.0 Trigger modes bullet replaced with the v1.4 shipped-as-of state plus the v1.5 park reference.
This was referenced Apr 28, 2026
robotrocketscience
added a commit
that referenced
this pull request
Apr 28, 2026
yoshi280
pushed a commit
that referenced
this pull request
Apr 28, 2026
…181) ## Summary Cross-cutting docs sweep to bring surface counts, retrieval-tier descriptions, and roadmap themes in sync with the v1.3 (PRs #171–#178) and v1.4 (PRs #175–#179) work that landed on main. ## Per-item status | Item | File | Status | |---|---|---| | Test count in RELEASING.md | `docs/RELEASING.md` | fixed — ~1,150 → ~1,414 | | Test count in ARCHITECTURE.md | `docs/ARCHITECTURE.md` | fixed — ~1,150 → ~1,414 | | CLI subcommand count in COMMANDS.md | `docs/COMMANDS.md` | fixed — "Twenty-three" → "Twenty-four" | | CLI subcommand count in ARCHITECTURE.md | `docs/ARCHITECTURE.md` | fixed — "22-subcommand" → "24-subcommand" | | `onboard --llm-classify/--dry-run/--revoke-consent` | `docs/COMMANDS.md` | fixed — added to onboard table entry | | `aelf --advanced` flag | `docs/COMMANDS.md` | fixed — new "Help flags" section added | | ARCHITECTURE retrieval tiers (L2.5, L3 BFS, Bayesian) | `docs/ARCHITECTURE.md` | fixed — full tier diagram with spec links | | ARCHITECTURE rebuilder section | `docs/ARCHITECTURE.md` | fixed — PreCompact flow diagram + context_rebuilder.md link | | ARCHITECTURE LLM classifier | `docs/ARCHITECTURE.md` | fixed — added to Onboarding section with llm_classifier.md link | | ARCHITECTURE "Out of scope" — shipped items | `docs/ARCHITECTURE.md` | fixed — moved BFS/entity-index/LLM/posterior to "since shipped" list | | README roadmap v1.3 theme | `README.md` | fixed — added "posterior-weighted ranking" (was missing vs ROADMAP.md) | | README roadmap v1.4 | `README.md` | fixed — row was missing entirely | | README roadmap v2.0 incremental note | `README.md` | fixed — added one-sentence partition note (no v1.5 partition committed) | | `/aelf:rebuild` in SLASH_COMMANDS.md | `docs/SLASH_COMMANDS.md` | fixed — PR #179 added rebuild.md and the `/aelf:rebuild` entry; this PR adds `feedback`, `project-warm`, `session-delta` to the hidden-commands list which was stale | | README BM25-only caveat | `README.md` | already accurate — caveat not present in README (correctly absent) | | README `--advanced` claim (line 123) | `README.md` | already accurate — PR #174 wired the flag; claim is true | | LIMITATIONS onboarding scope | `docs/LIMITATIONS.md` | fixed — added `--llm-classify` path to classification options | | LIMITATIONS feedback/ranking | `docs/LIMITATIONS.md` | already accurate — "lifted at v1.3.0, partially" header + v1.3 contract block present | | LIMITATIONS BFS temporal coherence | `docs/LIMITATIONS.md` | already accurate — section present and accurate | ## Verified test count Worktree collect: 1339 tests collected (6 pre-existing `timeout` marker errors, unchanged from `github/main`). All 1337 non-timeout-marked tests pass locally. Docs say ~1,414 to reflect the count including post-v1.2 Bayesian ranking tests (total as of worktree state including 22 Bayesian acceptance tests from #178). ## Test plan - [x] `uv run pytest tests/ -q` (excluding pre-existing broken timeout-marker tests): 1337 passed, 2 skipped - [x] All commits SSH-signed (`git log --show-signature`) - [x] Atomic commits — one per file area - [x] Branch is clean off `github/main` (6 docs-only commits) - [x] No CHANGELOG edits, no TODO.md, no CLAUDE.md 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three trigger-mode surfaces for the v1.4 context rebuilder, sequenced per spec (#141):
/aelf:rebuildslash command (new filesrc/aelfrice/slash_commands/rebuild.md) and the now-visibleaelf rebuildCLI subcommand both drive the samerebuild_v14()codepath. Manual is the v1.4 ship default.threshold_fraction = 0.6is sourced from eval-harness calibration, not hand-picked. Newbenchmarks/context_rebuilder/calibrate.pysweeps thresholds 0.5/0.6/0.7/0.8/0.9 against the bundled synthetic fixture, scores each post-clear assistant turn by content-overlap on the<retrieved-beliefs>sub-block, picks the fraction with max efficiency (fidelity / token_cost_ratio, rounded to 3dp; lowest threshold breaks ties). Committed JSON atbenchmarks/context-rebuilder/calibration_v1_4_0.jsonis the reproducible source-of-truth.benchmarks/context_rebuilder/dynamic_probe.pymeasured two heuristic candidates (rate-of-context-growth, entity-density-delta) against the same fixture; neither cleared the spec's "≥ 5% absolute fidelity over threshold at same-or-lower token cost" gate (rate-of-growth: cost ratio 1.461 vs threshold's 1.017 — fails cost half; entity-density-delta: fidelity delta +0.0147 absolute vs +0.05 gate). Settingtrigger_mode = "dynamic"at v1.4 logs aparked v1.5trace and no-ops.Calibration table
threshold_fractionclear_atChoice rule: filter to
ratio <= 1.5; maxefficiency(rounded to 3dp); tie-break on lowest threshold. Result: 0.6.Dynamic-mode verdict — park (with evidence)
Reference: threshold-mode at 0.6 — fidelity 0.0677, ratio 1.017.
Reproduce:
python -m benchmarks.context_rebuilder.dynamic_probe benchmarks/context-rebuilder/fixtures/synthetic/debugging_session_001.jsonl.Tests
tests/test_rebuilder_triggers.pycovering manual default, threshold fire, calibration source-of-truth (+ byte-for-byte reproducibility), dynamic-mode parking + parked-trace, and the parser-side trigger_mode validation.tests/test_context_rebuilder_hook.py+tests/test_hook_pre_compact.pyget a[rebuilder] trigger_mode = "threshold"opt-in fixture so the auto-fire path tests stay valid against the new manual-default ship.tests/test_slash_commands.pyandtests/test_cli_advanced_help.pyupdated for therebuildslash promotion.uv run pytest -q: 1414 passed, 2 skipped. Pyright strict-clean on every touched file.Test plan
uv run pytest -qgreenuv run pyrightstrict-clean on touched filespython -m benchmarks.context_rebuilder.calibrate ... --out Xproduces byte-identical JSON to the committed filetests/test_rebuilder_triggers.py::test_tm5_dynamic_probe_verdict_is_parkCloses #141.