diff --git a/CHANGELOG.md b/CHANGELOG.md index 5ae432f7c..385881252 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,20 +8,53 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 Pre-`1.0.0` releases are atomic milestones building toward the first installable release; see the roadmap in [README.md](README.md). -## [Unreleased] +## [1.6.0] - 2026-05-02 + +### Added + +- **Hook-hardening Phase 1 — framing-tag contract + content escape for memory blocks** ([#280](https://github.com/robotrocketscience/aelfrice/issues/280), [#297](https://github.com/robotrocketscience/aelfrice/pull/297)). Closes the prompt-injection surface where ingested belief content could forge or close the `` framing tag and inject downstream instructions into the host's prompt. Memory-block rendering now wraps each retrieved belief in a per-belief framing tag, escapes `<` / `>` / `&` in belief content at render time, and asserts the outer block tags match expected start/end on emit. Hosts that consume the framing tag can rely on it as a trust boundary. + +- **Hook-hardening mitigation 3 — per-turn audit log** ([#280](https://github.com/robotrocketscience/aelfrice/issues/280), [#314](https://github.com/robotrocketscience/aelfrice/pull/314)). New `/aelfrice/hook_audit.jsonl` records the exact rendered hook block on every fire of `user_prompt_submit` / `session_start` — telemetry records counts, audit records what was actually injected. Default-on, opt-out via `AELFRICE_HOOK_AUDIT=0` env or `[hook_audit] enabled = false` in `.aelfrice.toml`. 10 MB cap (configurable: `[hook_audit] max_bytes`); single-slot rotation to `hook_audit.jsonl.1` on rollover; fail-soft on I/O error. Read API: `aelfrice.hooks.audit.read_audit(path)`. 21 new tests in `tests/test_hook_audit.py`. + +- **`aelf tail` — live observability for hook injections** ([#321](https://github.com/robotrocketscience/aelfrice/issues/321), [#322](https://github.com/robotrocketscience/aelfrice/pull/322)). `tail -f`-style pretty-printer over the per-turn hook audit log. Streams each fire as a one-line summary (timestamp, session, n_locked, latency, prompt prefix); `--full` switches to the full rendered block. Reader half of the #321 observability story; complements the audit-log writer above. The audit record itself is extended with `beliefs[]` (per-belief id + tier), `latency_ms`, and `tokens` so the tail reader can surface what was injected at what cost without re-running retrieval. + +- **Belief retention class + per-source aging policy** ([#290](https://github.com/robotrocketscience/aelfrice/issues/290)). Phase-1 adds the `retention_class` schema column on `beliefs` (`unknown` | `transient` | `durable` | `permanent`) and the matching python validator. Phase-2 wires per-ingest-source defaults into `derive()` and the orphan-classifier scanner so new beliefs land with a non-`unknown` retention class based on which entry point ingested them. Phase-3 lands the promotion path: `MemoryStore.find_promotable_snapshots()` and `set_retention_class()` store APIs, surfaced through `aelf doctor --promote-retention` so an operator can run a one-shot pass that reclassifies low-prior beliefs based on retrieval / corroboration evidence accumulated since they were ingested. Foundational layer for the v2.0 aging / pruning policy spec'd in `docs/belief_retention_class.md`; no automated retention-driven eviction yet. + +- **Rebuild diagnostic log — phase-1a + phase-1c audit script** ([#288](https://github.com/robotrocketscience/aelfrice/issues/288), [#302](https://github.com/robotrocketscience/aelfrice/pull/302)). Phase-1a write path: every context rebuild emits a JSONL record under `/aelfrice/rebuild_logs/.jsonl` capturing the prompt, retrieval candidates per lane, dedupe stats, and the final pack-rate. Phase-1c reader: `scripts/audit_rebuild_log.py` summarises one or more session logs — pack-rate distribution (mean / p50 / p90), drop-reason histogram, per-lane contribution. Together they unblock the operator-week of in-tree evidence collection that gates the rebuild-redesign calibration work in #289 / #290 / #291. + +- **Posterior-ranking eval harness + heat-kernel composition** ([#151](https://github.com/robotrocketscience/aelfrice/issues/151), [#306](https://github.com/robotrocketscience/aelfrice/pull/306), [#310](https://github.com/robotrocketscience/aelfrice/pull/310)). Slice 1 (#306): new `benchmarks/posterior_ranking.py` harness measures MRR uplift and expected-calibration-error (ECE) of the posterior-ranked retrieval against the BM25 baseline on labeled fixtures. Slice 2 (#310): heat-kernel composition lands as a log-additive term in the ranking score; `use_heat_kernel` (default-OFF placeholder added in v1.5.0) is now wired through `retrieve_v2`. Default flip is gated on the harness clearing the MRR-uplift / ECE thresholds across the v2.0 corpus; see issue #151 for the gate criteria. + +- **Deferred-feedback sweeper — implicit retrieval-driven posterior signal** ([#191](https://github.com/robotrocketscience/aelfrice/issues/191), [#256](https://github.com/robotrocketscience/aelfrice/pull/256)). Background sweeper observes which retrieved beliefs are *referenced* by the host's subsequent turn (string-overlap signal against the next user prompt + assistant response) and emits implicit posterior-feedback events — `helped` when a retrieved belief shows up in the continuation, `noise` when retrieved-but-unreferenced beliefs accumulate. Feeds the `feedback_history` table that `aelf doctor` and the posterior-ranking harness consume. Replaces the v1.5 explicit-only feedback path with a default-on implicit signal; the explicit `aelf feedback` CLI remains for high-confidence overrides. + +- **v2.0 corpus public scaffold + bench-gate harness** ([#307](https://github.com/robotrocketscience/aelfrice/issues/307), [#311](https://github.com/robotrocketscience/aelfrice/pull/311), [#319](https://github.com/robotrocketscience/aelfrice/issues/319), [#320](https://github.com/robotrocketscience/aelfrice/pull/320)). Public-side scaffold for the v2.0 detector evaluation corpus: empty per-module directories under `tests/corpus/v2_0/` (dedup #197, enforcement #199, contradiction #201, wonder-consolidation #228, promotion-trigger #229, sentiment #193) plus the bench-gate harness in `tests/bench_gate/` that loads JSONL via the `corpus_root` fixture. The autouse `bench_gated` pytest marker skips when `AELFRICE_CORPUS_ROOT` is unset or the per-module dir is empty, so public CI stays green while the labeled corpus lives in the lab repo. Lab-side runner: `scripts/run_bench_gate.sh` exports the env var and runs the suite. + +- **`replay_full_equality` probe — flip-readiness gate for #262** ([#262](https://github.com/robotrocketscience/aelfrice/issues/262), [#304](https://github.com/robotrocketscience/aelfrice/pull/304)). New deterministic probe walks the append-only `ingest_log` (#205), replays every row through `derive()` into a fresh in-memory store, and asserts byte-equal equality against the live store on every public-API readout. Sentinel for the v2.0 view-flip: while the probe holds, the log-as-source-of-truth derivation can replace the legacy mutation-on-write path with no behaviour change. Fails loudly on any divergence; surfaces the first differing belief id + field for triage. + +- **Onboard / scanner / MCP `session_id` propagation to inserted beliefs (phantom-prereqs T3)** ([#192](https://github.com/robotrocketscience/aelfrice/issues/192)). Closes the third phantom-prereq from #190's session-tracking story: every ingest entry point now stamps `beliefs.session_id` so the corroboration recorder (#190) and the deferred-feedback sweeper (#191) can correlate retrieval signal with the originating session. `MemoryStore.scan_repo()` derives a synthetic per-invocation session id; the onboard-classifier path threads the active `onboard_sessions.session_id` through to insert; `tool_lock` (the MCP host-write entry point) accepts an optional `session_id` kwarg and tags the inserted belief with it. Beliefs without a discernible session continue to land with `session_id = NULL`; the column was already nullable. + +- **Reachable-install detection + multi-install upgrade warning** ([#345](https://github.com/robotrocketscience/aelfrice/issues/345)). New `lifecycle.detect_reachable_installs()` walks `PATH` and the standard `uv tool` / `pipx` venv roots to enumerate every aelfrice install reachable from the current shell. `aelf upgrade` now prints a warning block listing all reachable installs (with the active one marked) before upgrading, so users on multi-install machines see what they're about to update — and which siblings will silently keep running the old version. Eight new cases in `tests/test_upgrade_ux.py` cover the empty / single / multi cases. ### Changed - **Dev-only benchmark targets moved out of the `aelf` CLI** ([#342](https://github.com/robotrocketscience/aelfrice/issues/342)). `aelf bench verify-clean | longmemeval-score | posterior-residual` were always-failing in shipped wheels (the `benchmarks/` tree is dev-only and not packaged) and put dev-script knowledge in the runtime CLI surface. Replaced with module-level entry points runnable from a source checkout: `python -m benchmarks.verify_clean PATH ...`, `python -m benchmarks.longmemeval_score PREDS GT JUDGE`, `python -m benchmarks.posterior_ranking [--fixtures ... --seeds N --mrr-threshold X --ece-threshold Y --json --heat-kernel]`. The unknown-target error in `aelf bench` now points at the new entry points. The default `aelf bench` (synthetic harness in `src/aelfrice/benchmark.py`) and the `_BENCH_INERT_TARGETS` placeholders are unchanged. The deptry `DEP001 = ["benchmarks"]` ignore is removed — `src/aelfrice` no longer imports the dev-only package. +### Fixed + +- **Corroborations preserved across unique-applying migration** (refs [#336](https://github.com/robotrocketscience/aelfrice/issues/336)). The one-shot migration adding `UNIQUE(content_hash)` to `beliefs` did `DROP TABLE beliefs` with `PRAGMA foreign_keys=ON`, firing the `ON DELETE CASCADE` on `belief_corroborations` and wiping every corroboration row — including the synthetic `consolidation_migration` markers the dedup pass had just inserted. Wrap the swap in `PRAGMA foreign_keys=OFF/=ON` per the SQLite recommended pattern for table-mutation migrations. +- **`aelf doctor --gc-orphan-feedback` for dangling `feedback_history` rows** ([#223](https://github.com/robotrocketscience/aelfrice/issues/223), [#300](https://github.com/robotrocketscience/aelfrice/pull/300)). 100% of `feedback_history` rows had orphaned `belief_id`s after deletes; new doctor flag does a SQL delete-where-not-exists pass with a count printout. +- **Re-ingest row inflation — dedup by `content_hash`** ([#219](https://github.com/robotrocketscience/aelfrice/issues/219), [#283](https://github.com/robotrocketscience/aelfrice/pull/283)). Re-ingesting the same content under a different belief id created near-duplicates and inflated FTS5 retrieval ~5.3×; insert path now dedupes against `content_hash` and routes the second ingest to `record_corroboration` instead. +- **Rebuild block dedupes by `content_hash`** ([#281](https://github.com/robotrocketscience/aelfrice/issues/281), [#293](https://github.com/robotrocketscience/aelfrice/pull/293)). Symptom-level partial fix for the duplicate-output complaint in #281 — output stage drops same-content beliefs at render time so the host doesn't see two copies of the same fact. Full fix lands with the rebuild-redesign work in #289. +- **`step-security/harden-runner` SHA-pinned to v2.19.0 across workflows.** Replaces tag-pin with the corresponding 40-char commit SHA per repo convention; resolves Sourcery / CodeRabbit warnings on subsequent PRs. +- **Renamed `merge-conflict` label refs to `attn:merge-conflict`** in the `flag-stale-open-prs` workflow. The bare label didn't exist in this repo; four sites updated. + +## [1.5.1] - 2026-04-28 + ### Added - **Belief corroboration tracking — sibling table + ingest recorder (phantom-prereqs T1)** ([#190](https://github.com/robotrocketscience/aelfrice/issues/190)). New `belief_corroborations` table records each re-ingest of identical content without disturbing the existing dedup contract. When `ingest_turn` or `ingest_triples` encounters an already-existing belief (by id), it calls `MemoryStore.record_corroboration(belief_id, source_type=...)` instead of silently discarding the duplicate. Four `source_type` constants distinguish the ingest paths: `CORROBORATION_SOURCE_COMMIT_INGEST`, `CORROBORATION_SOURCE_TRANSCRIPT_INGEST`, `CORROBORATION_SOURCE_MCP_REMEMBER`, `CORROBORATION_SOURCE_HOOK_INGEST` (all in `aelfrice.models`); `record_corroboration` validates against `CORROBORATION_SOURCE_TYPES` and raises `ValueError` on unknown values. `Belief.corroboration_count` (default 0) is populated at retrieval time via a per-belief subquery in `get_belief`, `search_beliefs`, `search_beliefs_scored`, and `list_locked_beliefs`. `session_id` and `source_path_hash` columns are nullable; T3 (#192) will wire session propagation. `ON DELETE CASCADE` removes corroboration rows when a belief is deleted. 16 new deterministic tests in `tests/test_corroborations.py` cover schema creation, idempotency, re-ingest recording, corroboration count on retrieval, cascade deletion, nullable fields, source_type enum coverage, and ValueError on unknown source_type. - **Default-on LLM-Haiku onboard via host-driven classification — no API key required** ([#238](https://github.com/robotrocketscience/aelfrice/issues/238), [docs/llm_classifier.md](docs/llm_classifier.md), [docs/PRIVACY.md](docs/PRIVACY.md)). The typical aelfrice user already runs an LLM host (Claude Code, etc.); demanding a separate `ANTHROPIC_API_KEY` to opt into the v1.3.0 LLM-Haiku classifier pushed the higher-quality path out of reach for that population. v1.5.0 makes `/aelf:onboard` drive the four-class classifier through the host model's own Task tool against the smallest model in its stack — no API key, no separate billing, no MCP roundtrip. Quality matches the v1.3.0 LLM-Haiku ceiling (~12pp macro-F1 over the regex classifier on real prose); cost is sub-percent of a typical weekly host-plan allowance for one-time per-project onboard. Two new low-level CLI flags make the polymorphic handshake shell-callable: `aelf onboard --emit-candidates` persists a PENDING `onboard_sessions` row and prints `{session_id, n_already_present, sentences[]}` as JSON to stdout (no network); `aelf onboard --accept-classifications --session-id --classifications-file ` reads `[{index, belief_type, persist}, ...]` from a file or stdin, applies the host verdicts, and prints an `AcceptOnboardResult` JSON summary (no network). The slash command body at `src/aelfrice/slash_commands/onboard.md` is rewritten to (1) emit candidates via Bash, (2) batch up to 50 sentences per Task dispatch (`model=haiku`) with a four-class classification template, (3) aggregate the per-batch JSON arrays, and (4) feed the result into `--accept-classifications` via stdin. `--no-subagents` (or absence of the host's Task tool) falls through to the deterministic regex classifier. `[onboard.llm].enabled` flips default `False → True` with a soft-fallback in `check_gates`: users without the `[onboard-llm]` extra or without `ANTHROPIC_API_KEY` who haven't passed `--llm-classify` get the v1.0/v1.2 regex behaviour silently — explicit opt-in (`--llm-classify` or `--dry-run`) keeps the fail-fast install-hint path. Consent prompt copy expanded with cost ballpark, one-time scope, explicit data-sent / data-not-sent lists, the audit command, and the opt-out snippet. The direct-API CLI path (`aelf onboard --llm-classify`) remains the API-key-user fallback and is unchanged. 10 new in-process CLI tests in `tests/test_cli_onboard_handshake.py` cover the flag round-trip end-to-end (emit → accept via file and stdin → session marked COMPLETED, plus usage / unknown-session / bad-JSON failure modes). `tests/test_llm_classifier.py` updated for the new soft-fallback semantics (the no-SDK default-test-env case soft-falls instead of hard-failing). -## [1.5.1] - 2026-04-28 - ### Fixed - **`/aelf:upgrade` is now one-command-and-done ([#242](https://github.com/robotrocketscience/aelfrice/issues/242)).** Three independent bugs combined into a permanent "update available" banner that no upgrade could clear. (1) `lifecycle.installed_version()` read `aelfrice.__version__` from `src/aelfrice/__init__.py`, where the constant had been hardcoded to `"1.2.0"` since v1.2 and was never bumped on subsequent releases — so even after a successful upgrade to 1.5.0 the lifecycle module reported 1.2.0, the cache stayed `update_available=true`, and the orange banner kept firing forever. Fixed by reading `importlib.metadata.version("aelfrice")` instead, sourced from `pyproject.toml` at build time and tracking the published wheel automatically. `__init__.py:__version__` now also reads from package metadata; one source of truth. (2) `/aelf:upgrade` (and the underlying `aelf upgrade` CLI verb) hardcoded `pip install --upgrade aelfrice` for non-pipx, non-venv installs. For users on `uv tool install aelfrice` (the recommended install path) this was both wrong and unrunnable: pip is not on PATH in a uv-managed environment, so the suggestion failed outright. New `_is_uv_tool_install()` detection and a `uv tool upgrade aelfrice` branch in `upgrade_advice()`. (3) `/aelf:upgrade` only printed the command; the user had to copy-paste it into a terminal. Now it executes the upgrade in-process via `subprocess.run` for install methods where the upgrade is run by an external orchestrator (`uv tool` and `pipx`, both of which atomically swap the package and are safe to invoke from inside the running aelf binary). On success it wipes `~/.cache/aelfrice/update_check.json` so the orange banner clears immediately on the user's next prompt. Direct-pip installs still print the command (self-replacement of the running interpreter is unreliable on Windows); pass `--print-only` to force the pre-1.5.1 print-only behaviour on any install method. New flag: `--print-only`. `UpgradeAdvice` gains `argv: tuple[str, ...]` and `self_replacing: bool` fields. Slash command `aelf:upgrade` body updated to match the new behaviour. @@ -552,7 +585,8 @@ Foundation milestone — store, models, config. - Initial repo scaffold: pyproject, README, GitHub Actions workflows, scan configs (commit `67b4343`). -[Unreleased]: https://github.com/robotrocketscience/aelfrice/compare/v1.5.1...HEAD +[Unreleased]: https://github.com/robotrocketscience/aelfrice/compare/v1.6.0...HEAD +[1.6.0]: https://github.com/robotrocketscience/aelfrice/compare/v1.5.1...v1.6.0 [1.5.1]: https://github.com/robotrocketscience/aelfrice/compare/v1.5.0...v1.5.1 [1.5.0]: https://github.com/robotrocketscience/aelfrice/compare/v1.4.0...v1.5.0 [1.4.0]: https://github.com/robotrocketscience/aelfrice/compare/v1.2.0...v1.4.0 diff --git a/pyproject.toml b/pyproject.toml index 57a0fbd87..4670d4517 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "aelfrice" -version = "1.5.1" +version = "1.6.0" description = "Persistent memory for AI agents. Set up once. Stays out of your way. Local SQLite, auditable, no GPU, no network." readme = "README.md" requires-python = ">=3.12" diff --git a/uv.lock b/uv.lock index 0f3aa8e2d..7fa4887a9 100644 --- a/uv.lock +++ b/uv.lock @@ -12,7 +12,7 @@ resolution-markers = [ [[package]] name = "aelfrice" -version = "1.5.1" +version = "1.6.0" source = { editable = "." } dependencies = [ { name = "numpy" },