robotrocketscience · github-actions · May 12, 2026 · May 12, 2026 · May 12, 2026 · May 12, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -14,6 +14,8 @@ installable release; see the roadmap in [README.md](README.md).
 
 ### Added
 
+- **HRR persistence integration — docs bundle, closes #553 umbrella** ([#699](https://github.com/robotrocketscience/aelfrice/issues/699), closes [#553](https://github.com/robotrocketscience/aelfrice/issues/553)). Final sub-task of the HRR persistence umbrella. `docs/CONFIG.md` documents the `[retrieval] hrr_persist` TOML key (default `true`) and the `AELFRICE_HRR_PERSIST` env var override including the ephemeral-path auto-disable behaviour; `docs/COMMANDS.md` adds an *HRR persistence reporter* section explaining the three `aelf doctor` rows (`hrr.persist_enabled`, `hrr.on_disk_bytes`, `hrr.last_build_seconds`) and the `aelf status` `hrr.persist_state` one-line summary; `docs/feature-hrr-integration.md` acceptance-criteria checklist is updated to reflect shipped state with PR cross-references. Closes the five-PR HRR substrate: PR [#693](https://github.com/robotrocketscience/aelfrice/pull/693) (split-format save/load + cache mmap), PR [#701](https://github.com/robotrocketscience/aelfrice/pull/701) (ephemeral-path auto-disable, #695), PR [#703](https://github.com/robotrocketscience/aelfrice/pull/703) ([retrieval] hrr_persist TOML key, #698), PR [#704](https://github.com/robotrocketscience/aelfrice/pull/704) (`aelf doctor` reporter rows, #696), PR [#706](https://github.com/robotrocketscience/aelfrice/pull/706) (cold-start bench gate at N=50k, #697 — gate-shipped; bench-run capture stays open under `attn:bench-needed`). The `use_hrr_structural` default-flip remains out of scope for #553 and is gated separately on labelled-corpus evidence per the #154 composition tracker.
+
 - **Eval-judge κ runner module** ([#687](https://github.com/robotrocketscience/aelfrice/issues/687)). New `benchmarks/context_rebuilder/kappa.py` implements pure-stdlib Cohen's-κ computation for inter-judge agreement across N≥3 judge-response JSONL files (same shape as `read_judge_responses` in `llm_judge.py`), plus optional judge-vs-baseline κ against a substring-exact-match baseline file. `compute_kappa_report()` aligns turn_idx sets via set-intersection, computes pairwise κ across all run-pairs (key shape `<run_i>_vs_<run_j>`), and emits the `judge_kappa.json` artifact specified in `docs/BENCHMARKS.md §Eval-judge calibration`. The `calibrated` boolean gate requires `inter_judge_kappa.min ≥ 0.70` AND `hot_start_fidelity_mean ≥ 0.80` AND `n_runs ≥ 3` — judge-vs-baseline κ is reported but does not gate. `cohens_kappa()` handles the p_e=1 edge case (perfect-agreement raters) by returning 1.0 directly instead of dividing 0/0. CLI entry point at `python -m benchmarks.context_rebuilder.kappa --runs ... --baseline ... --out ...` returns exit 0 when calibrated and exit 1 otherwise for CI gating. Sixteen tests cover known-κ values, perfect-agreement / perfect-disagreement / single-class edges, intersection alignment when judge runs cover different turn_idx sets, calibration thresholds and failure-reason emission, and CLI smoke (calibrated → exit 0, n_runs<3 → exit 1).
 
 - **`[retrieval] hrr_persist` TOML key for HRR structural-index persistence** ([#698](https://github.com/robotrocketscience/aelfrice/issues/698)). Config-plumbing follow-up to #691 (persist-aware `HRRStructIndexCache`). Adds a `[retrieval] hrr_persist` boolean key to `.aelfrice.toml` that controls whether the HRR structural index is written to and loaded from `<store_dir>/.hrr_struct_index/`. Resolution precedence: `AELFRICE_HRR_PERSIST` env var (highest — "0"/"false"/"no"/"off" disables; "1"/"true"/"yes"/"on" forces on) → `[retrieval] hrr_persist` TOML key → default `true`. New `is_hrr_persist_enabled()` resolver in `retrieval.py` mirrors the `is_hrr_structural_enabled()` template. New `_env_hrr_persist_override()` helper + `ENV_HRR_PERSIST` / `HRR_PERSIST_FLAG` constants follow the same shape as the other flag constants. `HRRStructIndexCache` gains an optional `persist_enabled: bool | None = None` construction field; `_resolve_persist_dir()` now consults env first (exact truthy/falsy match), then the field, then the default-true path. New `make_hrr_struct_cache()` factory in `retrieval.py` is the canonical construction site: it calls `is_hrr_persist_enabled()` and passes the result into the cache so callers get config-driven persistence without manual flag resolution. Non-boolean `hrr_persist` TOML values trace to stderr and fall through to the default. Nine new unit tests and three integration tests cover every precedence path including the env-wins-in-both-directions invariant.

diff --git a/docs/COMMANDS.md b/docs/COMMANDS.md
@@ -36,9 +36,9 @@ DB resolves from `$AELFRICE_DB`, then `<git-common-dir>/aelfrice/memory.db` when
 |---|---|
 | `stats` | Belief / thread / lock / feedback counts. |
 | `health [--json]` | Structural auditor: orphan threads, FTS5 sync, locked contradictions, corpus volume. Includes a per-edge-type count breakdown (sorted by count desc, then alphabetically); empty store prints `no edges yet`. `--json` emits `{"audit": {...}, "features": {"edges_by_type": {...}}}`. Exits 1 on structural failure; corpus-volume warnings are informational. |
-| `status` | Alias for `health`. |
+| `status` | Alias for `health`. v3.0+ adds an `hrr.persist_state` summary line — see *HRR persistence reporter* below. |
 | `regime` | The v1.0 regime classifier output (`supersede` / `ignore` / `mixed` / `insufficient_data`). Informational; always exits 0. |
-| `doctor` | Verify hook + statusline commands resolve. Inspects `bash <script>` wrappers, flags `2>/dev/null \|\| true` patterns. Surfaces empty-store warning. Exits 1 on broken hooks. v1.6+ flags: `--gc-orphan-feedback` (delete `feedback_history` rows whose `belief_id` no longer exists, #223); `--promote-retention` (one-shot reclassification pass over low-prior beliefs based on accumulated retrieval / corroboration evidence, #290 phase-3). |
+| `doctor` | Verify hook + statusline commands resolve. Inspects `bash <script>` wrappers, flags `2>/dev/null \|\| true` patterns. Surfaces empty-store warning. Exits 1 on broken hooks. v1.6+ flags: `--gc-orphan-feedback` (delete `feedback_history` rows whose `belief_id` no longer exists, #223); `--promote-retention` (one-shot reclassification pass over low-prior beliefs based on accumulated retrieval / corroboration evidence, #290 phase-3). v3.0+ adds three HRR persistence rows (`hrr.persist_enabled`, `hrr.on_disk_bytes`, `hrr.last_build_seconds`) — see *HRR persistence reporter* below. |
 | `bench [--top-k N]` | Run the deterministic 16-belief × 16-query benchmark. Prints a JSON `BenchmarkReport`. |
 | `bench all --out PATH [--canonical] [--adapters CSV] [--smoke]` | (v2.0+, #437) Reproducibility harness — subprocess each academic-suite adapter (mab, locomo, longmemeval, structmemeval, amabench) at the canonical headline cut and merge into one schema-v2 JSON. `--canonical` asserts the run matches `CANONICAL_INVOCATIONS` (full benchmarks per the 2026-05-06 ratification) and refuses if the cut differs. `--smoke` runs the small SMOKE_INVOCATIONS subset. `--adapters` filters; combined with `--canonical` this refuses (cut mismatch). Returns 0 ok / 1 any error / 2 any skipped_data_missing. |
 | `tail [--full] [--since DUR] [--filter EXPR]` | (v1.6+) Live-tail the per-turn hook audit log. `tail -f`-style pretty-printer over `<git-common-dir>/aelfrice/hook_audit.jsonl`. Default one-line summary per fire (timestamp, session, n_locked, latency, prompt prefix); `--full` switches to the full rendered block. See [hook-injection-audit.md](hook-injection-audit.md). |
@@ -60,6 +60,35 @@ DB resolves from `$AELFRICE_DB`, then `<git-common-dir>/aelfrice/memory.db` when
 | `project-warm <path> [--debounce N]` | CwdChanged hook entry point. Resolves `<path>` to a project root (git work-tree or `~/.aelfrice/projects/<id>/`-provisioned ancestor), pre-loads the SQLite + OS page cache, and writes a sentinel under `~/.aelfrice/projects/<id>/.last_warm`. Silent no-op for unknown paths, denied paths (default deny: `/tmp/**`, `/var/folders/**`, `~/Downloads/**`, `~/Desktop/**` — override via `~/.aelfrice/config.json` `project_warm.deny_globs`), and any call inside the 60-second debounce window. Always exits 0; never writes to stdout. |
 | `session-delta [--id ID] [--telemetry-path PATH]` | **Advanced/hidden.** SessionEnd hook entry point. Computes per-session deltas (beliefs created, corrections detected, feedback given, velocity) from beliefs tagged with `--id` in the active store, combines with a current store snapshot (beliefs/graph blocks) and rolling-window rollups from the existing `telemetry.jsonl`, and appends one v=1 JSON row to `PATH` (default `~/.aelfrice/telemetry.jsonl`). Missing or empty `--id` is a silent no-op (stderr warning, exit 0). Idle sessions with zero beliefs still emit a row so `len(telemetry.jsonl)` equals session count. Not shown in `aelf --help`. |
 
+## HRR persistence reporter
+
+(v3.0+, #696) `aelf doctor` and `aelf status` report whether the HRR structural-index is persisted to disk, how large the blob is, and how long the most recent rebuild took. The rows let an operator confirm warm-load is actually firing instead of silently rebuilding every cold start.
+
+### `aelf doctor`
+
+Three rows under the `HRR` block:
+
+| Row | Type | Meaning |
+|---|---|---|
+| `hrr.persist_enabled` | `true` / `false` | Result of `HRRStructIndexCache._resolve_persist_dir()`. `true` when the resolver returns a non-`None` path — captures `store_path` set + `AELFRICE_HRR_PERSIST != "0"` + (#695) ephemeral-path not auto-disabled. |
+| `hrr.on_disk_bytes` | int | `os.path.getsize(<persist_dir>/struct.npy) + os.path.getsize(<persist_dir>/meta.npz)` when both files exist; `0` otherwise. |
+| `hrr.last_build_seconds` | float / `n/a` | Wall-clock of the most recent `HRRStructIndex.build()` call this process. `n/a` if no build has fired since process start. |
+
+`aelf doctor --json` adds the same three fields as a nested `hrr` object plus a `reason` string when persistence is off (`no store path`, `AELFRICE_HRR_PERSIST=0`, or `ephemeral path`).
+
+### `aelf status`
+
+One summary line:
+
+```
+hrr.persist_state: on <N> bytes, last build <X>s
+hrr.persist_state: off (<reason>)
+```
+
+Where `<reason>` is one of `no store path`, `AELFRICE_HRR_PERSIST=0`, `ephemeral path`, or `unknown (probe error)`.
+
+See [CONFIG § `hrr_persist`](CONFIG.md) for the underlying flag, [`docs/feature-hrr-integration.md`](feature-hrr-integration.md) for the substrate spec, and `tests/test_hrr_struct_index.py` for the matrix of observable states.
+
 ## Help flags
 
 `aelf --help` shows the everyday surface (visible subcommands). `aelf --help --advanced` (or `aelf --advanced`) shows the full surface including hidden subcommands (`bench`, `feedback`, `health`, `migrate`, `project-warm`, `regime`, `session-delta`, `stats`, `statusline`, `sweep-feedback`, `unsetup`). The `--advanced` flag was wired in v1.4 (PR #174).

diff --git a/docs/CONFIG.md b/docs/CONFIG.md
@@ -9,7 +9,7 @@ This is the reference for power users whose project has a documentation idiom or
 A single optional TOML file at the root of a project (or any ancestor). It exposes two power-user surfaces:
 
 - `[noise]` — onboard-time belief filter. Changes how `aelf onboard` ingests beliefs; nothing else.
-- `[retrieval]` (v1.3+) — retrieval-time tier toggles + ranking. Knobs: `entity_index_enabled` (L2.5), `bfs_enabled` (L3), `posterior_weight` (partial Bayesian-weighted L1 ranking), `use_bm25f_anchors` (BM25F-with-anchor-text since v1.7), `use_heat_kernel` (authority scoring lane, default-on since v2.1), `use_hrr_structural` (HRR structural-query lane, default-on since v2.1), `use_type_aware_compression` (per-belief retention-class compression, opt-in since v2.1). Two placeholder flags (`use_signed_laplacian`, `use_posterior_ranking`) are recognised but emit a deprecation warning if set — their lanes have not yet shipped.
+- `[retrieval]` (v1.3+) — retrieval-time tier toggles + ranking. Knobs: `entity_index_enabled` (L2.5), `bfs_enabled` (L3), `posterior_weight` (partial Bayesian-weighted L1 ranking), `use_bm25f_anchors` (BM25F-with-anchor-text since v1.7), `use_heat_kernel` (authority scoring lane, default-on since v2.1), `use_hrr_structural` (HRR structural-query lane, default-on since v2.1), `hrr_persist` (HRR structural-index on-disk persistence, default-on since v3.0), `use_type_aware_compression` (per-belief retention-class compression, opt-in since v2.1). Two placeholder flags (`use_signed_laplacian`, `use_posterior_ranking`) are recognised but emit a deprecation warning if set — their lanes have not yet shipped.
 
 Locks, hooks, MCP tools, and the Bayesian feedback math are not affected.
 
@@ -79,6 +79,16 @@ use_heat_kernel = true
 # AELFRICE_HRR_STRUCTURAL=0 env var overrides.
 use_hrr_structural = true
 
+# v3.0+. Default `true`. Persists the HRR structural-index
+# (struct.npy + meta.npz) to <store_dir>/.hrr_struct_index/ so
+# warm starts mmap the matrix instead of rebuilding (~38s at
+# N=50k → ~1s warm-load per #553). Auto-disabled when the store
+# root resolves under /tmp/, /var/tmp/, /dev/shm/, or /run/.
+# AELFRICE_HRR_PERSIST env var overrides (truthy/falsy match);
+# AELFRICE_HRR_PERSIST=1 forces persistence even on ephemeral
+# paths.
+hrr_persist = true
+
 # v2.1+. Default `false`, opt-in. Enables type-aware compression
 # (#434) — populates RetrievalResult.compressed_beliefs with per-
 # belief renderings keyed by retention_class (snapshot → headline,
@@ -315,6 +325,22 @@ Long-running consumers should pass an explicit `hrr_struct_index_cache: HRRStruc
 
 Precedence (first decisive wins): env var `AELFRICE_HRR_STRUCTURAL=0`/`1` > explicit Python kwarg `use_hrr_structural=<bool>` > TOML `[retrieval] use_hrr_structural` > default `true`. The flip landed when the #437 reproducibility-harness reached 11/11 (see #154). Set the flag to `false` for parity with the v2.0.x ranking.
 
+### `hrr_persist`
+
+Boolean, default `true` (v3.0+, #698). Toggles HRR structural-index persistence. When enabled, `HRRStructIndexCache` writes the built `(N, dim)` matrix to `<store_dir>/.hrr_struct_index/struct.npy` (plus the `meta.npz` metadata blob) on first build and `np.load(..., mmap_mode='r')`s it on every subsequent cold start — turning the ~38 s rebuild at N=50k into a ~1 s warm-load per `docs/feature-hrr-integration.md`. The save is atomic via temp-file + `os.replace` so readers never observe a partial write.
+
+**Ephemeral-path auto-disable** (#695). When the store root resolves under one of `/tmp/`, `/var/tmp/`, `/dev/shm/`, or `/run/`, the cache treats `hrr_persist` as if it were explicitly `false` and logs once per process:
+
+```
+aelfrice: HRR persistence disabled on ephemeral path <path>; set AELFRICE_HRR_PERSIST=1 to force.
+```
+
+Set `AELFRICE_HRR_PERSIST=1` to override the auto-disable. The TOML key cannot override (TOML lives at the store root which is itself the path being checked); the env var is the only escape hatch.
+
+Precedence (first decisive wins): env var `AELFRICE_HRR_PERSIST` (truthy `"1"`/`"true"`/`"yes"`/`"on"` forces on; falsy `"0"`/`"false"`/`"no"`/`"off"` disables) > explicit `persist_enabled=<bool>` on `HRRStructIndexCache(...)` > TOML `[retrieval] hrr_persist` > default `true`. Non-boolean TOML values trace to stderr and fall through to the default. The canonical construction site is `aelfrice.retrieval.make_hrr_struct_cache(...)`, which threads the resolved value into the cache for callers that don't manage flag resolution themselves.
+
+**When to disable.** Disk-constrained deployments (the on-disk blob is 328 MB at N=10k, 1.64 GB at N=50k; federation × multiple stores amplifies) and read-only filesystems are the two cases the opt-out exists for. Operators see the resolved state via `aelf doctor` (`hrr.persist_enabled` row) and `aelf status` (`hrr.persist_state` summary line).
+
 ### `use_type_aware_compression`
 
 Boolean, default `false`, opt-in (v2.1+, #434). Populates `RetrievalResult.compressed_beliefs` with per-belief renderings dispatched by `belief.retention_class`:

diff --git a/docs/feature-hrr-integration.md b/docs/feature-hrr-integration.md
@@ -183,16 +183,16 @@ The persistence work above is the **substrate** that makes default-ON viable. Th
 
 ## Acceptance criteria
 
-- [ ] `HRRStructIndex.save()` writes split format (`struct.npy` + `meta.npz`); existing tests pass against the new shape.
-- [ ] `HRRStructIndex.load(path)` accepts the split format AND the legacy bundled `.npz` (with deprecation log).
-- [ ] `HRRStructIndexCache` reads via `mmap_mode='r'` when the split format is present.
-- [ ] `aelf doctor` reports `hrr.persist_enabled`, `hrr.on_disk_bytes`, `hrr.last_build_seconds` rows.
-- [ ] Cold-start `aelf search` at N=50k synthetic store completes in ≤ 1 s when persisted (rebuilds in ≤ 38 s when not persisted; verifies `AELFRICE_HRR_PERSIST=0` codepath).
-- [ ] NDCG byte-equality test: build → save → load → probe vs in-memory probe must produce bit-identical scores.
-- [ ] `[retrieval] hrr_persist = false` in `.aelfrice.toml` disables persistence entirely; subsequent runs show no `.aelfrice/hrr/` directory created.
-- [ ] Ephemeral-path auto-disable: a store rooted at `/tmp/<x>/store` logs the documented message exactly once per process and creates no `.aelfrice/hrr/` directory; `AELFRICE_HRR_PERSIST=1` overrides this and creates the directory normally.
-- [ ] Cache invalidation: mutating a belief that participates in any HRR encoding drops both the in-memory index AND the on-disk blob.
-- [ ] `docs/CONFIG.md` and `docs/COMMANDS.md` document the new flags and `aelf doctor` rows.
+- [x] `HRRStructIndex.save()` writes split format (`struct.npy` + `meta.npz`); existing tests pass against the new shape. (PR [#693](https://github.com/robotrocketscience/aelfrice/pull/693))
+- [x] `HRRStructIndex.load(path)` accepts the split format AND the legacy bundled `.npz` (with deprecation log). (PR [#693](https://github.com/robotrocketscience/aelfrice/pull/693))
+- [x] `HRRStructIndexCache` reads via `mmap_mode='r'` when the split format is present. (PR [#693](https://github.com/robotrocketscience/aelfrice/pull/693))
+- [x] `aelf doctor` reports `hrr.persist_enabled`, `hrr.on_disk_bytes`, `hrr.last_build_seconds` rows. (PR [#704](https://github.com/robotrocketscience/aelfrice/pull/704))
+- [ ] Cold-start `aelf search` at N=50k synthetic store completes in ≤ 1 s when persisted (rebuilds in ≤ 38 s when not persisted; verifies `AELFRICE_HRR_PERSIST=0` codepath). *Bench-gate test shipped in PR [#706](https://github.com/robotrocketscience/aelfrice/pull/706); the gate skips under non-bench CI and requires an operator run with `AELFRICE_CORPUS_ROOT` set to capture the actual numbers. Tracked on [#697](https://github.com/robotrocketscience/aelfrice/issues/697) until the bench-needed flow produces a results artifact.*
+- [x] NDCG byte-equality test: build → save → load → probe vs in-memory probe must produce bit-identical scores. (`tests/test_hrr_struct_index.py::test_save_load_byte_identical_struct`, PR [#693](https://github.com/robotrocketscience/aelfrice/pull/693))
+- [x] `[retrieval] hrr_persist = false` in `.aelfrice.toml` disables persistence entirely; subsequent runs show no `.aelfrice/hrr/` directory created. (PR [#703](https://github.com/robotrocketscience/aelfrice/pull/703))
+- [x] Ephemeral-path auto-disable: a store rooted at `/tmp/<x>/store` logs the documented message exactly once per process and creates no `.aelfrice/hrr/` directory; `AELFRICE_HRR_PERSIST=1` overrides this and creates the directory normally. (PR [#701](https://github.com/robotrocketscience/aelfrice/pull/701))
+- [x] Cache invalidation: mutating a belief that participates in any HRR encoding drops both the in-memory index AND the on-disk blob. (`tests/test_hrr_struct_index.py::test_cache_invalidate_removes_disk_blob`, PR [#693](https://github.com/robotrocketscience/aelfrice/pull/693))
+- [x] `docs/CONFIG.md` and `docs/COMMANDS.md` document the new flags and `aelf doctor` rows. (PR [#699 — this bundle](https://github.com/robotrocketscience/aelfrice/issues/699))
 
 ---