Skip to content
Merged
2 changes: 2 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ Imports are one-directional — modules lower in the table import from higher.
| `CONTRADICTS` | -0.5 | half negative |
| `SUPERSEDES` | 0.0 | structural; no propagation |

The research line carried 17 edge types — 12 core (the six above plus `CALLS`, `TESTS`, `IMPLEMENTS`, `TEMPORAL_NEXT`, `CO_CHANGED`, `CONTAINS`, `COMMIT_TOUCHES`) and 5 speculative/causal (`SPECULATES`, `DEPENDS_ON`, `RESOLVES`, `HIBERNATED`, `DERIVED_FROM`). The narrowing to six is deliberate: the speculative/causal set hangs on the deferred `wonder` / multi-axis-uncertainty substrate (see [ROADMAP § Recovery inventory](ROADMAP.md#recovery-inventory)), and the additional core types (`CALLS`, `TESTS`, `IMPLEMENTS`, etc.) come back with the additional onboarding extractors that produce them — not by extending this enum in isolation.

**SQLite tables:** `beliefs`, `beliefs_fts` (virtual, porter unicode61), `edges` PK `(src, dst, type)`, `feedback_history`, `sessions`, `onboard_sessions`, `schema_meta`.

## Bayesian update
Expand Down
6 changes: 5 additions & 1 deletion docs/LIMITATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ This is a deliberate scope choice, not a roadmap item. Adding embeddings would b

The CLI scanner walks three sources: prose files (`*.md`, `*.rst`, `*.txt`, `*.adoc`), `git log`, and Python AST. Not yet wired: JavaScript / TypeScript / Rust / Go ASTs.

The research line shipped a wider set of extractors — citation-ref scanning across markdown bodies, test↔implementation linkage from filename and import patterns, and a directive detector that captured imperative user statements as TODO beliefs. v1.x does not ship these; the directive-detection path in particular has architectural implications (it would land alongside the violation-detection tier of [PHILOSOPHY § What we can and can't guarantee](PHILOSOPHY.md#what-we-can-and-cant-guarantee)) and is parked for v2.0.

Classification on the CLI path defaults to regex-based priors. Higher-quality classification is available via two paths:
- **MCP `aelf:onboard`** polymorphic flow, which routes through the host LLM.
- **`aelf onboard --llm-classify`** (v1.3+, default-off) — routes through Claude Haiku directly. Requires `ANTHROPIC_API_KEY`. Four consent gates enforce the privacy boundary. See [llm_classifier.md](llm_classifier.md).
Expand All @@ -33,7 +35,9 @@ The default retrieval mode (recall, not audit) is correctly served by latest-ser
## Sharp edges

- **Locks are durable.** Fresh lock = `(α, β) = (9.0, 0.5)`. Five independent contradicting feedback events are required to auto-demote. If you lock a wrong rule and correct it only once or twice, the lock keeps winning. `aelf demote <id>` drops it immediately.
- **`CONTRADICTS` edges drive demotion pressure.** The regex classifier rarely produces them, so demotion pressure accumulates only from manual feedback unless you wire commit-ingest or transcript-ingest.
- **`CONTRADICTS` edges drive demotion pressure.** The regex classifier rarely produces them, so demotion pressure accumulates only from manual feedback unless you wire commit-ingest or transcript-ingest. Note that this means the README-level "catches contradictions and flags them" claim is shipped *partially* at v1.x: contradiction **resolution** (`aelf resolve`, the tie-breaker that picks a winner given a CONTRADICTS edge) is in v1.0; contradiction **detection** (the post-insert semantic-divergence detector that creates those edges automatically) is deferred. Until detection lands, contradiction-flagging surfaces only on edges produced by the triple extractor's six explicit relation-family regexes.
- **Natural-language sentiment is not captured automatically.** Saying "ok good" or "no that's wrong" in chat does **not** strengthen or weaken the beliefs the agent just used. Feedback at v1.x requires explicit `aelf feedback <id> used|harmful` (CLI) or the equivalent MCP `aelf:feedback` call. The implicit-from-prose sentiment loop described in research-line marketing copy is a deferred capability, not a v1.x feature.
- **Confidence drops below 0.5 do not auto-flag.** A belief whose posterior drifts under the prior is not surfaced as a warning at v1.x. The only automatic state change driven by negative evidence is locked-belief demotion-pressure (≥5 contradictions → auto-demote). To find drifting beliefs, query `aelf stats` directly.
- **Jeffreys prior reads as 0.5.** A belief with no feedback reports posterior mean exactly `0.5`. That means "no evidence yet," not "coin-flip true."
- **`aelf onboard` is non-incremental on duplicates.** Re-runs are idempotent; existing beliefs are not re-scored or refreshed.
- **No bulk operations.** No batch lock, no `delete <pattern>`, no merge.
Expand Down
27 changes: 27 additions & 0 deletions docs/PHILOSOPHY.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,12 @@ harmful ⟹ β += 1

No embedding model. No hyperparameter search. No opaque ranking. Every score is one division and the audit trail is one table. The Bayesian update is itself one of the named rules traceability bottoms out in.

The posterior is **single-axis**. One `(α, β)` pair per belief, not a vector. A wrong belief is wrong overall; a useful belief is useful overall. The research line shipped a multi-axis `UncertaintyVector` (per-aspect `(α_i, β_i)` across existence / semantics / mechanism / cost) used by the speculative-belief surface (`wonder`, `reason`). aelfrice v1.x is single-axis only. Whether v2.0 adopts the multi-axis substrate is a load-bearing architectural decision tracked at [#196](https://github.com/robotrocketscience/aelfrice/issues/196); until that lands, assume single-axis when porting.

The cost: dense semantic similarity is gone. The benefit: a learning loop that converges on what works *for you*, not what's textually similar — and a retrieval pipeline that preserves determinism end to end.

What's intentionally absent: an exploration term in retrieval. The research-line requirement was that ≥15% of retrievals surface high-uncertainty beliefs to keep the feedback loop from collapsing into a filter bubble — confident beliefs reinforced, uncertain beliefs never re-tested. aelfrice does not yet address that requirement. Any exploration mechanism (bandit-style, entropy-weighted, sampling-based) breaks the "same query, same beliefs" property, which v1.x prioritises higher. v1.3 ships posterior reranking with no exploration term. If a future benchmark shows filter-bubble cost outweighs the determinism gain, exploration ships behind a flag in v2.x.

> At v1.0–v1.2 the posterior is computed and stored, but L1 retrieval still ranks by BM25 alone. The v1.3 retrieval wave wires the posterior into ranking. Until then, feedback updates the audit trail but doesn't yet move what the agent sees. See [LIMITATIONS](LIMITATIONS.md).

## Locks, not just decay
Expand All @@ -75,6 +79,8 @@ Hard locks ossify, though. So locks accumulate **demotion pressure** when contra

Your corrections live in one SQLite file on your machine. No cloud sync, no telemetry, no API calls in the retrieval path. The cloud LLM at the other end of your prompt sees whatever aelfrice injects — that's inherent — but aelfrice limits the slice (default 2,000 tokens, scoped to the current query) rather than dumping the whole memory.

The 2,000-token default is a calibrated choice, not an arbitrary one. The hypothesis from the research line is that focused context beats exhaustive context: a 2K-token retrieval that selects the right beliefs should match or exceed a 10K-token full-memory dump on response quality, while burning 5× fewer tokens on memory plumbing. The reproducibility cut at v2.0 is where that curve gets re-measured against the public retrieval pipeline; until then the 2,000-token budget is the inherited research-line default, configurable per-call.

[PRIVACY.md](PRIVACY.md) for verifiable specifics.

## Small surface, on purpose
Expand All @@ -89,6 +95,27 @@ Zero hard runtime dependencies. Python stdlib plus SQLite (the stdlib already wr

Every dependency is maintenance debt and attack surface. Heavier machinery — vector indices, embedding services, neural rerankers — earns its way in only when an experiment shows the existing stack is the bottleneck.

## What we can and can't guarantee

aelfrice is a memory substrate, not an LLM. The honest decomposition for any "the agent will follow this rule" claim:

| Tier | Mechanism | Guarantee |
|---|---|---|
| 1. Storage | SQLite WAL + locked belief | The rule is durably written and never lost. |
| 2. Injection | L0 always-loaded into every prompt | The rule is in the model's context on every retrieval. |
| 3. Compression survival | PreCompact rebuilder + locks-first ordering | The rule survives a context-window compaction. |
| 4. Violation detection | Not implemented at v1.x | — |
| 5. Violation blocking | Not implemented at v1.x | — |
| 6. LLM compliance | The model actually obeys the injected rule | **Not under aelfrice's control.** |

Tiers 1–3 hold mechanically. Tiers 4–5 (post-execution detection, pre-execution blocking) are research-line capabilities deferred to v2.x. Tier 6 is the LLM's own training and decoding, which aelfrice cannot constrain. If the model ignores an injected lock, the failure mode is in the model, not in aelfrice — but that distinction does not console a user whose agent just ran `git push` despite a clear directive.

Two recovery angles fall out of the same substrate:

- **Session recovery, not just write durability.** SQLite WAL guarantees that every acknowledged write survives a crash. That is the storage-engine claim. The product-level claim is that the *working context* of an interrupted session is reconstructable on restart — not from a snapshot file, but from the same belief store the next session retrieves against. Re-open the terminal next week, ask "where were we?", and the locks plus recent retrieval-relevant beliefs are still there.
- **Confidence does not auto-flag.** A belief whose posterior drifts below 0.5 is not surfaced as a warning at v1.x. Only locked-belief demotion-pressure (≥5 contradictions → auto-demote) produces a visible state change. If you want to know which beliefs are losing the feedback loop, you ask `aelf stats`; the system does not interrupt to tell you.
- **The append-only substrate at v1 is `feedback_history`, not observations.** The research line had a separate `observations` table that was insert-only — every observation that produced a belief was permanently recorded. aelfrice v1 does not have that table. Beliefs are the substrate, and beliefs *are* mutated (decay adjusts age weighting; feedback updates `(α, β)`). What aelfrice v1 *does* keep append-only is `feedback_history`: every `apply_feedback` event writes a row, and rows are never updated. That is the immutable substrate at v1, and it is sufficient for "did the user actually correct this?" audit. Full ingest-log immutability — recording every observation that produced or refreshed a belief, not only every feedback event — is the v2.0 contract; see [`design/write-log-as-truth.md`](design/write-log-as-truth.md) for the proposed table and the migration story.

## What this design buys

- **Continuity.** Close the terminal, come back next week, "where were we?" — the memory restores it.
Expand Down
28 changes: 27 additions & 1 deletion docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,33 @@ What the research line had, when each piece returns:
| HRR vocabulary bridge | v2.0.0 |
| Type-aware compression | v2.0.0 |
| Doc / semantic linker | v2.0.0 |
| `wonder` / `reason` / `core` / `unlock` / `delete` / `confirm` | v2.0.0 |
| Graph-traversal store methods (`expand_graph`, `get_neighbors`, `edge_exists`) | v1.5 or v1.6 (substrate for `wonder` + `reason`; ships ahead of v2.0) |
| `ingest_turn(bulk=)` parameter | v2.0.0 ([#194](https://github.com/robotrocketscience/aelfrice/issues/194); mechanical, post-`wonder_ingest` port) |
| `scoring.uncertainty_score(α, β)` | v2.0.0 ([#195](https://github.com/robotrocketscience/aelfrice/issues/195); conditional on substrate decision) |
| Multi-axis uncertainty substrate (`UncertaintyVector`) | v2.0.0 substrate decision ([#196](https://github.com/robotrocketscience/aelfrice/issues/196); load-bearing — blocks `wonder` + `reason`) |
| Speculative-belief schema migration (3 columns + 1 belief type + 2 edge types) | v2.0.0 (depends on #196) |
| Speculative / causal edge types (`SPECULATES`, `DEPENDS_ON`, `RESOLVES`, `HIBERNATED`) | v2.0.0 (with `wonder`) |
| `wonder` (gap-analysis frontend) | v2.0.0 (depends on substrate + graph-traversal) |
| `wonder_ingest` + `wonder_gc` (speculative-belief lifecycle) | v2.0.0 (depends on substrate) |
| `reason` (graph-walk reasoning) | v2.0.0 (depends on graph-traversal) |
| `core` / `unlock` / `delete` / `confirm` (CLI surface) | v2.0.0 |
| Directive-detection + compliance-audit + selective-injection triad | v2.0.0 candidate ([#199](https://github.com/robotrocketscience/aelfrice/issues/199)) |
| Sentiment-from-prose feedback | v2.0.0 candidate ([#193](https://github.com/robotrocketscience/aelfrice/issues/193)) |
| Near-duplicate audit (`aelf doctor dedup`) | v1.x candidate ([#197](https://github.com/robotrocketscience/aelfrice/issues/197)) |
| Multi-model belief classifier (SIGNAL/NOISE/STALE/CONTESTED) | v2.0.0 candidate ([#198](https://github.com/robotrocketscience/aelfrice/issues/198)) |
| Automatic CONTRADICTS detection (semantic-divergence) | v1.x candidate ([#201](https://github.com/robotrocketscience/aelfrice/issues/201)) |

The four "candidate" lines are the orphaned research-line capabilities from the agentmemory parity audit — neither shipping today nor previously listed on this roadmap. They land if and only if a benchmark or experiment justifies the inclusion (per the validation discipline below); otherwise they stay parked.

### Deliberately not on this list

The research line also shipped the following capabilities that aelfrice does **not** plan to recover:

- **Research-artifact provenance metadata** (`produced_at` / `method` / `sample_size` / `data_source` / `independently_validated` per belief) and the **rigor-tier** classification layer (`hypothesis` / `simulated` / `empirically_tested` / `validated`). Motivated in the research line by a case study where a new agent miscalibrated project maturity from raw completion counts. aelfrice v1 stores provenance via the `Belief.origin` enum (7 source-tier values) only; epistemic-rigor metadata is **not** on the v2.0 surface. If status reporting needs this signal it lands as a separate feature with its own benchmark, not as a schema-wide migration.
- **Session-velocity tracking** (items/hour decay scaling). v1 ships per-belief decay with type-specific half-lives; velocity-scaled decay is the research-line refinement and is parked.
- **Calibrated status reporting** that surfaces rigor-tier distribution and velocity context to a new agent. Depends on the two items above.
- **Cross-project shared scopes** via SQLite ATTACH. Subsumed by the Multi-project query non-goal in [LIMITATIONS § Sharing, sync, or federation](LIMITATIONS.md). Named here so the research-line term ("shared scopes") doesn't read as an oversight.
- **Obsidian vault export** and **vault-as-source-of-truth** storage. Rejected at v1: SQLite is the source of truth, per-project isolation is a hard property. Subsumed by the same federation non-goal.

## Compatibility

Expand Down
Loading