fix(ledger): route ledger_sync deserialization warnings to wipe-and-replay recovery (#301)

CHANGELOG.md

@@ -3,6 +3,24 @@
 All notable changes to bicameral-mcp are tracked here. Format loosely follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+## v0.15.1 — hotfix: route `ledger_sync` deserialization warnings to a wipe-and-replay recovery path (#301)
+
+Hotfix for [#301](https://github.com/BicameralAI/bicameral-mcp/issues/301). v0.15.0 already added a row-level deserialization probe to `bicameral.diagnose` ([`cli/_diagnose_gather.py::_probe_row_deserialization`](cli/_diagnose_gather.py)), but the probe's findings stopped at the `suggestions` list — `_classify_recovery` in `handlers/diagnose.py` still inspected only `schema_meta.version`, so an agent that ran `diagnose` after a `link_commit` failure would see `recovery_path: "clean"` and `next_action: "No remediation needed."` while the ledger was actually unreadable.
+
+This release closes the loop end-to-end:
+
+- **`LedgerDeserializationError`** (subclass of `LedgerError`) is raised from `ledger.client.query` / `ledger.client.execute` whenever the SurrealDB SDK returns `Invalid revision \`N\` for type \`Value\`` or a `deserialization error` wrapper. The exception message embeds the recovery command, so the agent sees the wipe-and-replay instruction inside the MCP error envelope without needing a second `diagnose` round-trip.
+- **`handlers/diagnose.py::_classify_recovery`** now consults `Diagnosis.row_probe_warnings` *before* the schema-version checks. Non-empty warnings route to `reset_rebuild` (when `.bicameral/events/*.jsonl` is present) or `reset_destructive` (no events on disk) with a `next_action` that quotes the exact `bicameral_reset(wipe_mode="ledger", replay_from_events=…, confirm=True)` call.
+- **`handlers/sync_middleware.py::ensure_ledger_synced`** re-raises `LedgerDeserializationError` instead of swallowing it at DEBUG. The broad `except Exception` is still in place for transient catch-up failures (the original "best-effort" contract); only deserialization errors break out, because they're the one class of failure the agent must surface to the user.
+
+### Fixed
+
+- **#301** — `bicameral.link_commit` no longer fails with a bare `LedgerError("SurrealDB rejected query: ...")` when `ledger_sync` rows can't be deserialized; the new exception class names the recovery path. `bicameral.diagnose` now classifies the same failure mode under `reset_rebuild` / `reset_destructive` so the agent's next_action is actionable.
+
+### Notes for v0.14.x → v0.15.x upgraders hit by #301
+
+The SurrealDB SDK pin (`surrealdb==2.0.0`) is unchanged between v0.14.x and v0.15.1; upgrading the package does NOT rewrite the persisted SurrealKV record format. An installation that hit the `Invalid revision \`3\` for type \`Value\`` error on v0.14.x will continue to see the same SDK-level deserialization failure on first `link_commit`. What changes in v0.15.1 is the *visibility*: the error now surfaces with an explicit recovery command instead of as a generic `LedgerError`, and `diagnose` correctly routes to a reset path. To recover, run `bicameral_reset(wipe_mode="ledger", replay_from_events=True, confirm=True)` (if `.bicameral/events/*.jsonl` is present) or `bicameral_reset(wipe_mode="ledger", confirm=True)` otherwise.
+
 ## v0.15.0 — PII archive, hard-delete `remove_decision`, schema v17→v24 chain, team-mode foundations
 
 Cumulative release draining the dev → main backlog accumulated since v0.14.7. Lands the **#221 PII archive** (operator-erasable PII surface), retires the **soft-delete tombstone model** for `bicameral.remove_decision` (now hard-delete by default), brings the constant-time **`bicameral_meta.decision_revision` counter** (#87) into the preflight dedup path, ships **`bicameral.admin/query`** and **dashboard source view** (#278 Phase 1+3), wires the **`LocalDirectorySourceAdapter`** and **`sync-and-brief`** team-mode flows (#344, #279), and adds the **code-locator singleton + eager startup init** that moves index work off the MCP stdio handshake (#243, #380).

RECOMMENDED_VERSION

@@ -1 +1 @@
-0.15.0
+0.15.1

handlers/diagnose.py

@@ -121,8 +121,26 @@ def _classify_recovery(diagnosis) -> tuple[RecoveryPath, str]:
     exp = diagnosis.schema_version_expected
     has_events = _events_present(diagnosis.ledger_url)
 
-    if rec is not None and rec > exp:
+    # #301 — row-level deserialization warnings outrank the schema-version
+    # check. Schema can read clean (`schema_meta` is a tiny row, never the
+    # one that breaks) while the operational tables hit
+    # ``Invalid revision `N` for type Value``. The probe in
+    # cli/_diagnose_gather.py::_probe_row_deserialization catches that; here
+    # we route the verdict so the agent sees recovery_path=reset_rebuild
+    # instead of "clean, no remediation needed".
+    row_warnings: list[str] = list(getattr(diagnosis, "row_probe_warnings", []) or [])
+    if row_warnings:
         path: RecoveryPath = "reset_rebuild" if has_events else "reset_destructive"
+        tables = ", ".join(sorted({w.split(":", 1)[0] for w in row_warnings}))
+        return path, (
+            f"Row-level deserialization warnings on {tables} — likely a "
+            "SurrealDB embedded-SDK record-format mismatch. Run "
+            f"`bicameral_reset(wipe_mode='ledger', replay_from_events={has_events}, "
+            "confirm=True)` to wipe and replay from .bicameral/events/."
+        )
+
+    if rec is not None and rec > exp:
+        path = "reset_rebuild" if has_events else "reset_destructive"
         return path, (
             f"Ledger schema v{rec} is newer than this binary (v{exp}). "
             f"Upgrade `bicameral-mcp` to a version that understands v{rec}, "

handlers/sync_middleware.py

@@ -20,6 +20,8 @@
 from datetime import UTC, datetime
 from typing import TYPE_CHECKING
 
+from ledger.client import LedgerDeserializationError
+
 if TYPE_CHECKING:
     from contracts import LinkCommitResponse, SessionStartBanner
 
@@ -288,6 +290,14 @@ async def ensure_ledger_synced(ctx) -> LinkCommitResponse | None:
             _LAST_SYNCED_SHA = live_head
             logger.debug("[sync_middleware] catch-up ran for %s", live_head[:8])
             return result
+    except LedgerDeserializationError:
+        # #301 — row-format mismatch on ledger_sync / yields / etc. This is
+        # the *opposite* of a swallowable error: the ledger is in a state
+        # the user must repair before any tool will work. Re-raise so the
+        # MCP transport layer surfaces the exception (with embedded
+        # RECOVERY_HINT) to the agent instead of silently logging at DEBUG.
+        logger.warning("[sync_middleware] ledger row-format mismatch — surfacing to agent")
+        raise
     except Exception as exc:
         logger.debug("[sync_middleware] catch-up failed: %s", exc)
     return None

ledger/client.py

@@ -115,6 +115,55 @@ class LedgerError(RuntimeError):
     """
 
 
+class LedgerDeserializationError(LedgerError):
+    """Raised when SurrealDB's embedded SDK can't decode a row on read (#301).
+
+    The on-disk SurrealKV record header carries a revision number that must
+    match the surrealdb-py deserializer. A mismatch surfaces as
+    ``Invalid revision `N` for type `Value```. This is *below* the schema
+    layer — `init_schema`/`migrate` don't help because the migration code
+    can't read the row either. The fix is to wipe the affected cursor and
+    replay from the event log via
+    ``bicameral_reset(wipe_mode="ledger", replay_from_events=True, confirm=True)``.
+
+    Subclass of ``LedgerError`` so existing ``except LedgerError`` handlers
+    catch it; callers that surface a recovery hint to the agent match on
+    ``LedgerDeserializationError`` specifically.
+    """
+
+    RECOVERY_HINT = (
+        "Row-level deserialization failed — likely a SurrealDB embedded SDK "
+        "revision mismatch on persisted rows. Run "
+        "`bicameral_reset(wipe_mode='ledger', replay_from_events=True, "
+        "confirm=True)` to wipe and replay from .bicameral/events/, or "
+        "`bicameral-mcp diagnose` for a full report."
+    )
+
+    def __init__(self, *, raw: str, sql_prefix: str) -> None:
+        self.raw = raw
+        self.sql_prefix = sql_prefix
+        super().__init__(
+            f"SurrealDB row deserialization failed: {raw}\n"
+            f"SQL: {sql_prefix}\n"
+            f"Recovery: {self.RECOVERY_HINT}"
+        )
+
+
+_DESERIALIZATION_SIGNATURES = ("Invalid revision", "deserialization error")
+
+
+def _is_deserialization_error(raw: str) -> bool:
+    """Return True when ``raw`` looks like a SurrealKV row-format mismatch.
+
+    Matches the two signatures observed in #301 and the related ``yields``
+    incident: the SDK's ``Invalid revision `N` for type `T``` and the more
+    general ``A deserialization error occured`` wrapper. Both come back as
+    strings the caller sees as ``LedgerError`` today — this helper is the
+    classification seam.
+    """
+    return any(sig in raw for sig in _DESERIALIZATION_SIGNATURES)
+
+
 class LedgerTimeoutError(LedgerError):
     """Raised when a ledger query exceeds its wallclock timeout budget.
 
@@ -270,8 +319,13 @@ async def query(
         try:
             result = await self._run_with_timeout(sql, vars, timeout_class)
         except SurrealError as exc:
+            msg = str(exc)
+            if _is_deserialization_error(msg):
+                raise LedgerDeserializationError(raw=msg, sql_prefix=sql[:300]) from exc
             raise LedgerError(f"SurrealDB rejected query: {exc}\nSQL: {sql[:300]}") from exc
         if isinstance(result, str):
+            if _is_deserialization_error(result):
+                raise LedgerDeserializationError(raw=result, sql_prefix=sql[:300])
             raise LedgerError(f"SurrealDB rejected query: {result}\nSQL: {sql[:300]}")
         return _normalize(result) if isinstance(result, list) else []
 
@@ -297,8 +351,13 @@ async def execute(
         try:
             result = await self._run_with_timeout(sql, vars, timeout_class)
         except SurrealError as exc:
+            msg = str(exc)
+            if _is_deserialization_error(msg):
+                raise LedgerDeserializationError(raw=msg, sql_prefix=sql[:300]) from exc
             raise LedgerError(f"SurrealDB rejected statement: {exc}\nSQL: {sql[:300]}") from exc
         if isinstance(result, str):
+            if _is_deserialization_error(result):
+                raise LedgerDeserializationError(raw=result, sql_prefix=sql[:300])
             raise LedgerError(f"SurrealDB rejected statement: {result}\nSQL: {sql[:300]}")
 
     async def execute_many(

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "bicameral-mcp"
-version = "0.15.0"
+version = "0.15.1"
 description = "Decision ledger MCP server — ingests meeting transcripts, maps decisions to code, tracks drift"
 readme = "README.md"
 requires-python = ">=3.10"