feat(doctor): nag when per-project DBs are on pre-v1.x schema (#589)

CHANGELOG.md

@@ -16,6 +16,8 @@ installable release; see the roadmap in [README.md](README.md).
 
 ### Added
 
+- **`aelf doctor` flags per-project DBs on pre-v1.x schema (no `origin` column)** ([#589](https://github.com/robotrocketscience/aelfrice/issues/589)). Scans all `~/.aelfrice/projects/*/memory.db` and appends a `legacy-schema per-project DBs detected` block listing each flagged DB with belief row count and idle days. DBs on the old schema cannot participate in the v2.x lifecycle (`agent_remembered`, `user_validated`, calibrated weights, `aelf:promote`). Block is quiet when no legacy DBs are found (parity with #557 quietness). `aelf migrate` is now visible in `--help` output (previously `argparse.SUPPRESS`) so the fix line in the nag block is discoverable.
+
 - **Context-rebuilder eval harness — three integration points wired** ([#592](https://github.com/robotrocketscience/aelfrice/issues/592)). `benchmarks/context-rebuilder/eval_harness.py` was a skeleton with five `NotImplementedError` stubs since v1.2.0; this lands the three pure-Python wirings that make `--mode threshold-sweep` and `--mode budget-sweep` runnable end-to-end. `replay_to_fork` populates a fresh in-memory `MemoryStore` via `ingest_jsonl` against a tempfile holding the first `fork_turn` lines of the case JSONL. `run_rebuilder` calls `rebuild_v14()` with `floor_session=floor_l1=trigger_threshold` (mapping the harness's threshold-sweep axis onto the v1.7 per-lane composite-score floor) and returns `(rebuilt_block, latency_ms)` measured via `time.monotonic()`. `measure_token_cost` returns `estimate_tokens(rebuilt) / estimate_tokens(pre_clear)` using `benchmarks.context_rebuilder.measure.estimate_tokens` (the 4-chars-per-token heuristic that mirrors `aelfrice.context_rebuilder._CHARS_PER_TOKEN`), so harness measurements stay aligned with the rebuilder's own budget bookkeeping. `replay_post_fork` is a non-crashing stub that returns one placeholder per `eval_turn` (`{turn_idx, expected, actual="", matched=False, reason="needs_replay_client"}`) so the harness produces valid latency + token-cost numbers before the model-invocation client lands; `score_fidelity` reads `matched=False` and returns 0.0 instead of raising. Adds `debugging_session_001.meta.json` next to the bundled 16-turn synthetic fixture (midpoint fork at turn 8, eval_turns at indices 8/10/12/14) so `--corpus benchmarks/context-rebuilder/fixtures/synthetic/` resolves out of the box. 15 new tests in `tests/test_context_rebuilder_eval_harness_wiring.py` cover each wired function plus an end-to-end threshold-sweep smoke against the bundled fixture. The model-invocation client + LLM judge stages are explicit follow-ups; this commit lands the harness's runnable contract before fidelity scoring engages.
 
 - **Session-end Stop hook prompts to lock session corrections** ([#582](https://github.com/robotrocketscience/aelfrice/issues/582)). New `aelf-stop-hook` (default-on) fires once per assistant-turn end, walks the store for unlocked correction-class beliefs created in the current `session_id`, and emits a `<aelfrice-session-end>` block to stderr listing each candidate with a pre-filled `aelf lock --statement '<...>'` command. Candidate filter: `session_id == current AND lock_level != LOCK_USER AND (type == BELIEF_CORRECTION OR origin in {agent_inferred, agent_remembered})`. Hook is informational by default; setting `AELF_AUTOLOCK_CORRECTIONS=1` in the environment makes it auto-lock the candidates instead (logs each lock to stderr for transparency). Wired into `aelf setup` / `aelf unsetup` (`--no-stop-hook` opts out) and into `aelf doctor` as a fourth default-on auto-capture hook the v2.1 nag flags when missing. Coexists with the existing transcript-ingest Stop entry as a separate entry under the same `hooks.Stop` event key. **Note**: the issue's spec assumed a `aelf correct` CLI command and a `feedback_history.kind=correct` marker that don't exist on `main` (no historical implementation); the v0 detection signal is correction-class beliefs in the current session instead. Future work to ship `aelf correct` + `feedback_history.kind` would let this hook also surface beliefs that were *modified* (not only newly-created) in the session.

docs/INSTALL.md

@@ -170,6 +170,29 @@ All hooks are non-blocking. Every failure path returns exit 0 — a hook problem
 
 > **Privacy note.** Default-on transcript-ingest means every turn you type lands in the per-project SQLite DB on `PreCompact` rotation. The DB is local-only (no network, no telemetry — see § "Your data stays yours" in the README) but the JSONL has no PII scrubber. If you paste secrets, customer data, or anything you don't want indexed in chat, opt out with `--no-transcript-ingest` and use `aelf lock` / `aelf onboard` for explicit ingestion only.
 
+### Legacy-schema detection (`aelf doctor`, v2.1+)
+
+`aelf doctor` scans all per-project DBs under `~/.aelfrice/projects/*/memory.db` and flags any that use the pre-v1.x schema (no `origin` column on the `beliefs` table) and have at least one row. DBs on the old schema cannot participate in the v2.x lifecycle — `agent_remembered`, `user_validated`, calibrated weights, `aelf:promote` — because the column that tracks origin is absent.
+
+When legacy DBs are found the doctor report appends a block like:
+
+```
+legacy-schema per-project DBs detected (pre-v1.x, no `origin` column).
+  ~/.aelfrice/projects/2e7ed55e017a/memory.db (35,332 beliefs, idle 16d)
+  ~/.aelfrice/projects/18a856c7a96b/memory.db (6,283 beliefs, idle 13d)
+fix: `aelf migrate --from <path> --apply` per DB to copy beliefs
+     into the current project's modern-schema DB.
+```
+
+The block is quiet when every scanned DB already has the `origin` column. Empty DBs (zero rows) are silently skipped.
+
+To migrate a legacy DB:
+
+```bash
+aelf migrate --from ~/.aelfrice/projects/<id>/memory.db          # dry-run
+aelf migrate --from ~/.aelfrice/projects/<id>/memory.db --apply  # write
+```
+
 ---
 
 ## Update notifier

src/aelfrice/cli.py

@@ -4140,8 +4140,10 @@ def build_parser(*, show_advanced: bool = False) -> argparse.ArgumentParser:
     p_regime = sub.add_parser("regime", help=argparse.SUPPRESS)
     p_regime.set_defaults(func=_cmd_regime)
 
-    # Hidden: one-shot v1.0 -> v1.1 migration; the era is over.
-    p_migrate = sub.add_parser("migrate", help=argparse.SUPPRESS)
+    p_migrate = sub.add_parser(
+        "migrate",
+        help="copy beliefs from a legacy per-project DB into the current one",
+    )
     p_migrate.add_argument(
         "--from", dest="from_path", default=None,
         help="legacy DB path (default: ~/.aelfrice/memory.db)",

src/aelfrice/doctor.py

@@ -40,6 +40,8 @@
 import re
 import shlex
 import shutil
+import sqlite3
+import time
 from dataclasses import dataclass, field
 from pathlib import Path
 from typing import TYPE_CHECKING, Final, Literal, cast
@@ -223,6 +225,26 @@ def _load_settings_json(path: Path) -> dict[str, object]:
 # How many trailing lines of the hook-failures log to surface.
 _HOOK_FAILURES_TAIL: Final[int] = 10
 
+# Root directory under which per-project aelfrice state lives.
+# Each sub-directory is a project-id slug; memory.db sits directly inside.
+# Override via `aelfrice_projects_dir` kwarg on `diagnose()` (tests use this).
+_AELFRICE_PROJECTS_DIR: Final[Path] = (
+    Path.home() / ".aelfrice" / "projects"
+)
+
+
+@dataclass(frozen=True)
+class LegacySchemaDB:
+    """One per-project DB detected as pre-v1.x (no `origin` column).
+
+    `path`      — absolute path to the memory.db file.
+    `row_count` — number of rows in the `beliefs` table.
+    `idle_days` — whole days since the file was last modified (mtime).
+    """
+    path: Path
+    row_count: int
+    idle_days: int
+
 
 @dataclass(frozen=True)
 class CommandFinding:
@@ -292,6 +314,14 @@ class DoctorReport:
     missing_auto_capture_hooks: list[str] = field(
         default_factory=lambda: cast(list[str], [])
     )
+    # Per-project DBs under ~/.aelfrice/projects/*/memory.db that use
+    # the pre-v1.x schema (no `origin` column on the `beliefs` table)
+    # and have at least one row. These DBs cannot participate in the
+    # v2.x lifecycle (agent_remembered, user_validated, calibrated
+    # weights, aelf:promote) without `aelf migrate` (#589).
+    legacy_schema_dbs: list[LegacySchemaDB] = field(
+        default_factory=lambda: cast(list[LegacySchemaDB], [])
+    )
 
     @property
     def broken(self) -> list[CommandFinding]:
@@ -324,6 +354,7 @@ def diagnose(
     known_cli_subcommands: frozenset[str] | None = None,
     search_tool_telemetry_path: Path | None = None,
     user_prompt_submit_telemetry_path: Path | None = None,
+    aelfrice_projects_dir: Path | None = None,
 ) -> DoctorReport:
     """Walk user and project settings.json, return a DoctorReport.
 
@@ -339,6 +370,8 @@ def diagnose(
     derivable from the project root's git-common-dir), the
     search_tool_hook telemetry section is populated. Similarly for
     `user_prompt_submit_telemetry_path` (#218 AC4).
+    `aelfrice_projects_dir` overrides the default scan root for
+    per-project DBs (`~/.aelfrice/projects`); useful in tests (#589).
     """
     user_path = user_settings if user_settings is not None else USER_SETTINGS_PATH
     project_path = (
@@ -358,6 +391,13 @@ def diagnose(
     report.hook_failures_tail = _tail_log(log_path, _HOOK_FAILURES_TAIL)
     report.missing_runtime_deps = _check_runtime_deps()
     report.missing_auto_capture_hooks = _check_auto_capture_hooks(report.findings)
+    # #589: scan per-project DBs for pre-v1.x schema (no `origin` column).
+    _proj_dir = (
+        aelfrice_projects_dir
+        if aelfrice_projects_dir is not None
+        else _AELFRICE_PROJECTS_DIR
+    )
+    report.legacy_schema_dbs = _check_legacy_schema_dbs(projects_dir=_proj_dir)
     if known_cli_subcommands is not None:
         slash_dir = (
             slash_commands_dir if slash_commands_dir is not None
@@ -748,6 +788,7 @@ def format_report(report: DoctorReport) -> str:
     _format_user_prompt_submit_telemetry_section(report, lines)
     _format_missing_runtime_deps_section(report, lines)
     _format_missing_auto_capture_section(report, lines)
+    _format_legacy_schema_section(report, lines)
     return "\n".join(lines)
 
 
@@ -811,6 +852,89 @@ def _format_missing_auto_capture_section(
     )
 
 
+def _check_legacy_schema_dbs(
+    *,
+    projects_dir: Path | None = None,
+) -> list[LegacySchemaDB]:
+    """Return per-project DBs that are on the pre-v1.x schema (no `origin`).
+
+    Scans `~/.aelfrice/projects/*/memory.db` (or `projects_dir` override).
+    A DB is flagged when ALL of the following hold:
+      1. The `beliefs` table exists.
+      2. No column named `origin` is present.
+      3. `SELECT COUNT(*) FROM beliefs` returns > 0.
+
+    Opens each DB read-only (`file:...?mode=ro`) to avoid accidental writes.
+    Any DB that raises a connection or query error is silently skipped —
+    doctor is diagnostic only.
+    """
+    root = projects_dir if projects_dir is not None else _AELFRICE_PROJECTS_DIR
+    if not root.is_dir():
+        return []
+
+    results: list[LegacySchemaDB] = []
+    now = time.time()
+
+    for db_path in sorted(root.glob("*/memory.db")):
+        try:
+            uri = f"file:{db_path}?mode=ro"
+            con = sqlite3.connect(uri, uri=True)
+        except Exception:  # noqa: BLE001
+            continue
+        try:
+            cur = con.execute("PRAGMA table_info(beliefs)")
+            columns = cur.fetchall()
+            if not columns:
+                # No beliefs table — skip (uninitialised or unrelated DB).
+                continue
+            col_names = {row[1] for row in columns}  # row[1] is the column name
+            if "origin" in col_names:
+                # Modern schema — skip.
+                continue
+            # Legacy schema. Skip empty DBs — uninteresting.
+            (row_count,) = con.execute("SELECT COUNT(*) FROM beliefs").fetchone()
+            if row_count == 0:
+                continue
+            mtime = db_path.stat().st_mtime
+            idle_days = int((now - mtime) / 86400)
+            results.append(
+                LegacySchemaDB(
+                    path=db_path,
+                    row_count=int(row_count),
+                    idle_days=idle_days,
+                )
+            )
+        except Exception:  # noqa: BLE001
+            pass
+        finally:
+            con.close()
+
+    return results
+
+
+def _format_legacy_schema_section(
+    report: DoctorReport, lines: list[str],
+) -> None:
+    """Append the pre-v1.x legacy-schema nag block (#589) to `lines`.
+
+    Quiet when no legacy DBs are found (parity with #557 quietness).
+    """
+    if not report.legacy_schema_dbs:
+        return
+    lines.append("")
+    lines.append(
+        "legacy-schema per-project DBs detected (pre-v1.x, no `origin` column)."
+    )
+    for entry in report.legacy_schema_dbs:
+        lines.append(
+            f"  {entry.path} ({entry.row_count:,} beliefs, idle {entry.idle_days}d)"
+        )
+    lines.append(
+        "fix: `aelf migrate --from <path> --apply` per DB to copy beliefs "
+        "into the current project's modern-schema DB."
+    )
+
+
 def _format_missing_runtime_deps_section(
     report: DoctorReport, lines: list[str],
 ) -> None: