Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ installable release; see the roadmap in [README.md](README.md).

### Added

- **`aelf doctor --prune-dormant` interactively deletes idle per-project DBs** ([#594](https://github.com/robotrocketscience/aelfrice/issues/594)). Companion to the #589 legacy-schema nag: scans `~/.aelfrice/projects/*/memory.db` for files whose mtime is older than `--idle-days` (default 30) and lists them with row count + size + idle days. Default action is dry-run; `--apply` prompts `[y/N]` per DB and deletes only on `y`/`yes`. There is no `--yes` shortcut — every deletion is per-DB and explicit, locking the issue body's "never silent delete" constraint. Detection is schema-agnostic (legacy and modern DBs are both pruneable when idle), and empty DBs / unreadable schemas surface with `row_count=0` rather than being skipped — an idle empty DB is the cleanest prune target. The filesystem-orphan detection mode listed in the issue body (project root no longer exists) is deferred: `project_warm.py` hashes the git-common-dir to a 12-char slug and stores no reverse mapping, so rebuilding `id → original-path` is a separate piece of work. `--projects-dir PATH` overrides the scan root for tests.

- **`aelf doctor` flags per-project DBs on pre-v1.x schema (no `origin` column)** ([#589](https://github.com/robotrocketscience/aelfrice/issues/589)). Scans all `~/.aelfrice/projects/*/memory.db` and appends a `legacy-schema per-project DBs detected` block listing each flagged DB with belief row count and idle days. DBs on the old schema cannot participate in the v2.x lifecycle (`agent_remembered`, `user_validated`, calibrated weights, `aelf:promote`). Block is quiet when no legacy DBs are found (parity with #557 quietness). `aelf migrate` is now visible in `--help` output (previously `argparse.SUPPRESS`) so the fix line in the nag block is discoverable.

- **Context-rebuilder eval harness — opt-in LLM-judge stage for open-ended turns** ([#592](https://github.com/robotrocketscience/aelfrice/issues/592)). Commit-3 of #592. New `benchmarks/context-rebuilder/judges/llm_judge.py` adds a host-polymorphic judge that scores open-ended replay rows (those tagged `reason="needs_llm_judge"` by the deterministic substring path). `write_judge_requests(rows, run_dir, max_judge_calls=N)` writes `judge_requests.jsonl` carrying strictly `(turn_idx, expected, actual)` — the rebuilt block and user turn are deliberately excluded per `docs/BENCHMARKS.md` Pass 1 / Pass 2 separation; letting the judge see retrieval context would inflate fidelity. Default `max_judge_calls=0` disables the stage so CI runs are free and outbound model traffic is opt-in. The judge tier label is `"anchor"` — the operator maps this to their host CLI's calibration-baseline tier (the tier the prior Cohen's-κ disagreement methodology was measured at); a cheaper-tier knob is intentionally not exposed because cheaper judges weaken comparability on the open-ended turns this stage exists to score. Dispatch is operator-driven: the host CLI reads the request file, issues one off-band model call per row at the anchor tier with `JUDGE_PROMPT_TEMPLATE`, writes `judge_responses.jsonl`. `read_judge_responses` + `apply_judge_verdicts` then fold verdicts back into the replay rows (clearing `reason`, setting `matched`, adding `judge_rationale`). No provider SDK import; matches `/aelf:onboard`'s polymorphic-classify pattern. 15 deterministic tests in `tests/test_context_rebuilder_eval_judge.py` cover the contamination boundary, cost-cap binding, response round-trip, malformed-line skip, fold-verdicts purity, and the no-SDK guard. Harness-side wiring (so the stage rides on `eval_harness.py --run-dir` rather than the standalone helpers) joins onto PR #601's run-dir plumbing in a follow-up.
Expand Down
24 changes: 24 additions & 0 deletions docs/INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,30 @@ aelf migrate --from ~/.aelfrice/projects/<id>/memory.db # dry-run
aelf migrate --from ~/.aelfrice/projects/<id>/memory.db --apply # write
```

### Pruning dormant per-project DBs (`aelf doctor --prune-dormant`, v2.1+)

Some per-project DBs hold beliefs from projects you worked on briefly with an older aelfrice version, then abandoned. They never get migrated, never get touched, and just sit there. `aelf doctor --prune-dormant` lists DBs whose `memory.db` mtime is older than `--idle-days` (default 30) and lets you delete them one at a time.

```bash
aelf doctor --prune-dormant # dry-run: list dormant DBs only
aelf doctor --prune-dormant --idle-days 90 # tighter idle threshold
aelf doctor --prune-dormant --apply # prompts [y/N] per DB; deletes on 'y'
```

Sample output:

```
found 2 dormant per-project DB(s) (idle >= 30d, 41,615 beliefs, 18,432.0 KiB total):
~/.aelfrice/projects/2e7ed55e017a/memory.db (35,332 beliefs, 15,120.4 KiB, idle 35d)
~/.aelfrice/projects/18a856c7a96b/memory.db (6,283 beliefs, 3,311.6 KiB, idle 32d)

dry-run only. re-run with `--apply` to be prompted [y/N] per DB.
```

`--apply` prompts per DB; the default at the prompt is N, so anything other than `y`/`yes` (including a bare Enter or piping `/dev/null`) preserves the file. There is no `--yes` shortcut — destructive deletion is always per-DB and explicit. Unlike `aelf migrate`, this does not move beliefs anywhere; it only removes the DB file.

The dormant scan is schema-agnostic — both pre-v1.x and modern-schema DBs are flagged when idle. A DB you still want to migrate should go through `aelf migrate --from <path> --apply` (above) before pruning, not after.

---

## Update notifier
Expand Down
144 changes: 144 additions & 0 deletions src/aelfrice/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,9 @@
from aelfrice.derivation import DerivationInput, derive
from aelfrice.derivation_worker import run_worker
from aelfrice.doctor import (
DORMANT_IDLE_DAYS_DEFAULT,
DormantDB,
_check_dormant_dbs,
classify_orphans as _classify_orphans,
diagnose,
format_orphan_feedback_report as _format_orphan_feedback_report,
Expand Down Expand Up @@ -3050,6 +3053,8 @@ def _cmd_doctor(args: argparse.Namespace, out: object) -> int:
return _cmd_doctor_detect_stale(args, out)
if getattr(args, "derive_pending", False):
return _cmd_doctor_derive_pending(args, out)
if getattr(args, "prune_dormant", False):
return _cmd_doctor_prune_dormant(args, out)
scope = getattr(args, "scope", None)
exit_code = 0
if scope in (None, "hooks"):
Expand Down Expand Up @@ -3450,6 +3455,111 @@ def _cmd_doctor_promote_retention(
return 0


def _cmd_doctor_prune_dormant(
args: argparse.Namespace, out: object
) -> int:
"""Identify per-project DBs idle for >= --idle-days (default 30)
and optionally delete them after explicit per-DB confirmation (#594).

Default action is dry-run: scan + list. With ``--apply``, the
user is prompted ``[y/N]`` per DB; only ``y``/``Y`` deletes the
file. Anything else preserves the DB and proceeds to the next
one. Bypasses the hooks/graph checks; never opens the canonical
store.

Honours ``--idle-days N`` (override default), ``--projects-dir``
(override scan root, primarily for tests).
"""
idle_days_raw = getattr(args, "idle_days", None)
idle_days = (
int(idle_days_raw) if idle_days_raw is not None
else DORMANT_IDLE_DAYS_DEFAULT
)
if idle_days < 0:
print(
f"aelf doctor --prune-dormant: --idle-days must be >= 0 (got {idle_days})",
file=sys.stderr,
)
return 2

projects_dir_raw = getattr(args, "projects_dir", None)
projects_dir = Path(projects_dir_raw) if projects_dir_raw else None

apply = bool(getattr(args, "apply", False))

dormant = _check_dormant_dbs(
projects_dir=projects_dir, idle_days=idle_days,
)
if not dormant:
print(
f"no per-project DBs idle for >= {idle_days} days.",
file=out, # type: ignore[arg-type]
)
return 0

total_bytes = sum(d.size_bytes for d in dormant)
total_beliefs = sum(d.row_count for d in dormant)
print(
f"found {len(dormant)} dormant per-project DB(s) "
f"(idle >= {idle_days}d, {total_beliefs:,} beliefs, "
f"{total_bytes / 1024:.1f} KiB total):",
file=out, # type: ignore[arg-type]
)
for entry in dormant:
print(_format_dormant_entry(entry), file=out) # type: ignore[arg-type]

if not apply:
print(
"",
file=out, # type: ignore[arg-type]
)
print(
"dry-run only. re-run with `--apply` to be prompted [y/N] per DB.",
file=out, # type: ignore[arg-type]
)
return 0

deleted = 0
skipped = 0
for entry in dormant:
prompt = f"delete {entry.path}? [y/N] "
try:
answer = input(prompt).strip().lower()
except EOFError:
answer = ""
if answer in ("y", "yes"):
try:
entry.path.unlink()
deleted += 1
print(f" deleted {entry.path}", file=out) # type: ignore[arg-type]
except OSError as exc:
print(
f" ERROR deleting {entry.path}: {exc}",
file=sys.stderr,
)
skipped += 1
else:
skipped += 1
print(f" kept {entry.path}", file=out) # type: ignore[arg-type]

print("", file=out) # type: ignore[arg-type]
print(
f"prune complete: {deleted} deleted, {skipped} kept.",
file=out, # type: ignore[arg-type]
)
return 0


def _format_dormant_entry(entry: DormantDB) -> str:
"""One-line summary of a dormant DB for the prune listing."""
return (
f" {entry.path} "
f"({entry.row_count:,} beliefs, "
f"{entry.size_bytes / 1024:.1f} KiB, "
f"idle {entry.idle_days}d)"
)


def _print_doctor_store_check(out: object) -> None:
"""Surface the 'is this store populated?' check (issue #116).

Expand Down Expand Up @@ -4331,6 +4441,40 @@ def build_parser(*, show_advanced: bool = False) -> argparse.ArgumentParser:
"hooks/graph checks."
),
)
p_doctor.add_argument(
"--prune-dormant",
dest="prune_dormant",
action="store_true",
default=False,
help=(
"list per-project DBs under ~/.aelfrice/projects/*/memory.db "
"that have been idle for >= --idle-days (default 30). "
"Combine with --apply to be prompted [y/N] per DB and "
"delete on 'y'. Bypasses the hooks/graph checks; never "
"opens the canonical store. Issue #594."
),
)
p_doctor.add_argument(
"--idle-days",
dest="idle_days",
type=int,
default=None,
metavar="N",
help=(
"with --prune-dormant: idle threshold in days (default: 30). "
"Files with mtime older than now - N days qualify."
),
)
p_doctor.add_argument(
"--projects-dir",
dest="projects_dir",
default=None,
metavar="PATH",
help=(
"with --prune-dormant: override the per-project DB scan "
"root (default: ~/.aelfrice/projects). Primarily for tests."
),
)
p_doctor.add_argument(
"--max-drift",
dest="max_drift",
Expand Down
95 changes: 95 additions & 0 deletions src/aelfrice/doctor.py
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,24 @@ class LegacySchemaDB:
idle_days: int


@dataclass(frozen=True)
class DormantDB:
"""One per-project DB detected as idle for >= the dormancy threshold (#594).

`path` — absolute path to the memory.db file.
`row_count` — number of rows in the `beliefs` table (0 when the
table is missing or empty; both still count as
pruneable when the file itself is dormant).
`idle_days` — whole days since the file was last modified (mtime).
`size_bytes` — file size of the memory.db, surfaced so the user
knows what storage they reclaim by pruning.
"""
path: Path
row_count: int
idle_days: int
size_bytes: int


@dataclass(frozen=True)
class CommandFinding:
"""One hook/statusline command we inspected.
Expand Down Expand Up @@ -912,6 +930,83 @@ def _check_legacy_schema_dbs(
return results


# Default minimum idle period before a per-project DB is treated as
# dormant by `_check_dormant_dbs` (#594). Conservative: a project a
# user touches monthly stays out of the prune list. Override via the
# `idle_days` kwarg on the scanner (CLI: `--idle-days N`).
DORMANT_IDLE_DAYS_DEFAULT: Final[int] = 30


def _check_dormant_dbs(
*,
projects_dir: Path | None = None,
idle_days: int = DORMANT_IDLE_DAYS_DEFAULT,
) -> list[DormantDB]:
"""Return per-project DBs whose mtime is at least `idle_days` days old.

Scans `~/.aelfrice/projects/*/memory.db` (or `projects_dir`
override). A DB is flagged when ALL of the following hold:
1. The memory.db file exists and stat() succeeds.
2. (now - mtime) / 86400 >= `idle_days`.

Schema is irrelevant — both legacy and modern DBs can be dormant.
Empty DBs (zero beliefs, or no `beliefs` table at all) are still
flagged when dormant; an idle empty DB has zero forward value and
is the cleanest prune target.

Opens each DB read-only (`file:...?mode=ro`) for the row-count
probe; any DB whose row-count probe fails (corrupt, no `beliefs`
table) is reported with `row_count=0` rather than skipped — the
file is still a pruneable artefact even when the schema is unreadable.
"""
root = projects_dir if projects_dir is not None else _AELFRICE_PROJECTS_DIR
if not root.is_dir():
return []

results: list[DormantDB] = []
now = time.time()
threshold_seconds = idle_days * 86400

for db_path in sorted(root.glob("*/memory.db")):
try:
stat = db_path.stat()
except OSError:
continue
age_seconds = now - stat.st_mtime
if age_seconds < threshold_seconds:
continue

row_count = 0
try:
uri = f"file:{db_path}?mode=ro"
con = sqlite3.connect(uri, uri=True)
except Exception: # noqa: BLE001
con = None
if con is not None:
try:
cur = con.execute("PRAGMA table_info(beliefs)")
if cur.fetchall():
(rc,) = con.execute(
"SELECT COUNT(*) FROM beliefs"
).fetchone()
row_count = int(rc)
except Exception: # noqa: BLE001
pass
finally:
con.close()

results.append(
DormantDB(
path=db_path,
row_count=row_count,
idle_days=int(age_seconds / 86400),
size_bytes=int(stat.st_size),
)
)

return results


def _format_legacy_schema_section(
report: DoctorReport, lines: list[str],
) -> None:
Expand Down
Loading
Loading