From bb81fa7180706b9dbcccabd9fee2fe96f9e1e7fc Mon Sep 17 00:00:00 2001 From: Gerard Slee Date: Thu, 2 Jul 2026 11:04:26 -0400 Subject: [PATCH 01/74] docs(mobile): add standalone offline-first mobile app plan Viability exploration + phased roadmap for an Expo mobile companion: - standalone offline-first app with its own SQLite DB - synced-folder oplog transport (never syncs the binary cairn.db) - 2 personal devices, single user; LWW + conflict-copy (no CRDTs) - online-only chat via Rork toolkit + Vercel AI SDK - MVP scope: notes, board, search, chat, lightweight knowledge graph --- docs/plans/mobile-app-viability.md | 286 +++++++++++++++++++++++++++++ 1 file changed, 286 insertions(+) create mode 100644 docs/plans/mobile-app-viability.md diff --git a/docs/plans/mobile-app-viability.md b/docs/plans/mobile-app-viability.md new file mode 100644 index 000000000..eba55910d --- /dev/null +++ b/docs/plans/mobile-app-viability.md @@ -0,0 +1,286 @@ +# Cairn Mobile App — Viability Exploration & Plan + +Status: Draft / exploration +Owner: — +Date: 2026-07-02 +Related: `docs/architecture-review.md`, `electron/db/schema.ts`, `electron/shared/notes-io.ts`, `electron/file-watcher.ts`, `electron/db/queries.ts` + +--- + +## 1. Goal & confirmed requirements + +Ship a **standalone, offline-first** mobile app (iOS + Android) for Cairn, built in **Expo**, +that maintains **one shared codebase brain** with the desktop app. + +Confirmed with product owner: + +- **Standalone app.** NOT a remote control of the desktop. It must work with the desktop app closed. +- **Offline-first.** The app has its **own local database** and syncs bidirectionally with the desktop workspace. +- **MVP UI scope:** Notes (read + edit), Board (view + move cards), Search, Chat. Plus a **lightweight Knowledge Graph** view. Explicitly **out**: Idea Flow, Insights, full desktop Knowledge Graph. +- **Sync transport: synced-folder oplog** (no backend, local-first). *(decided)* +- **Scope: 2 personal devices** — single user, no multi-user concurrency. Simplifies the conflict model. *(decided)* +- **Chat is online-only** via the **Rork AI toolkit** (`POST {EXPO_PUBLIC_TOOLKIT_URL||https://toolkit.rork.com}/text/llm/`), using the **Vercel AI SDK** (`ai`, already a Cairn devDep) with a custom provider. Chat is the one feature that degrades when offline. *(decided)* + +This is the harder of the two architectures considered in the earlier draft. Choosing +standalone + offline means **we are committing to building a real sync engine**, because we +now have two independently-writable databases that must reconcile without data loss. + +--- + +## 2. The core challenge: sync, not UI + +The UI rebuild in Expo is bounded, well-understood work. The **risk and cost concentrate in +the sync engine**. The codebase research below establishes exactly what the current data +model gives us for free and what we must add. + +### What we get for free (sync-friendly facts) + +| Fact | Why it helps | Source | +|---|---|---| +| **All IDs are client-generated `nanoid(12)`** (~71 bits) | Two devices can create rows offline with negligible collision risk. The hardest part of distributed creation is *already solved*. | `electron/db/utils.ts:6,9-11` | +| **Notes already dual-write to `.md` files** | Notes have a portable, mergeable text representation independent of the binary DB. | `electron/shared/notes-io.ts:179` | +| **A working FS→DB reconciler already exists** | The chokidar file-watcher already imports external `.md` edits into the DB with echo-suppression — a partial sync precedent. | `electron/file-watcher.ts` | +| **`updated_at` is set on every write** for most tables (via `ts()`) | A "changed since X" watermark is partially buildable today; indexes `idx_notes_updated_at` / `idx_cards_updated_at` exist. | `electron/db/utils.ts:14`, `schema.ts:163,165` | +| **`version` counters exist on `notes` + `task_cards`** | A starting point for optimistic concurrency / conflict detection. | `schema.ts:308,317` | + +### What is missing and MUST be added (the actual project) + +| Gap | Consequence if ignored | Required addition | +|---|---|---| +| **No tombstones. Deletes are hard-deletes** (except `archived_at` soft-archive on workspaces/projects/notes/cards) | A diff-based sync **cannot tell "deleted on peer A" from "not-yet-created on peer B"** → deleted items resurrect. `board_columns`, `tags`, `chat_*`, and all idea-flow tables have no soft-delete at all. | `deleted_at`/tombstone rows on every syncable table. | `schema.ts` (see §6) | +| **No changelog / oplog. `db:changed` is a zero-payload signal** ("something changed", re-hydrate everything) | No way to ask "what changed and how" — can't build incremental sync. | A changelog table (entity, op, logical clock, hash) populated by a write-wrapper or triggers. | `registry.ts:83-89` | +| **`tags` has NO timestamps at all** | Tag create/rename/recolor/delete is invisible to any timestamp diff. | Add `created_at`, `updated_at`, `version`, `deleted_at` to `tags`. | `schema.ts:93-98` | +| **`version` is advisory, not enforced** — the renderer write path never checks it; only some MCP tools do, and they return a soft error, never a hard reject | Cannot rely on it as a compare-and-swap gate. | Promote to real CAS (`WHERE id=? AND version=?`) on the write path, or adopt HLC. | `queries.ts:215,357`; `mcp/tools/notes.ts:126-131` | +| **Timestamps mix trusted (`ts()`) and untrusted (frontmatter `updatedAt`, file mtime) sources; cross-device clock skew** | Pure wall-clock watermarks corrupt under skew. | **Hybrid Logical Clocks (HLC)** or Lamport timestamps instead of trusting ISO strings. | `notes-files.ts:292`; `notes-io.ts:125-135` | +| **Existing note reconciliation is last-writer-wins by timestamp, whole-record, no field merge, no conflict record** | Concurrent edits silently lose one side. | A defined conflict policy + conflict surfacing (see §5). | `notes-files.ts:119-135` | + +**Bottom line:** the transport and the collision-free ID scheme are reusable. The +replication protocol (changelog + tombstones + logical clock + CAS + conflict policy) is +net-new and is the crux of the project. + +--- + +## 3. Recommended architecture + +**One shared TypeScript core, two thin UI shells, and a sync layer that both DBs speak.** + +``` + ┌───────────────────────────────────────────┐ + │ @cairn/core (shared package) │ + │ domain types · zod schemas · store logic │ + │ sync protocol (changelog, HLC, tombstones, │ + │ conflict resolution) · repository interface│ + └───────────────┬───────────────┬─────────────┘ + │ │ + ┌────────────────────────▼──┐ ┌─────▼───────────────────────┐ + │ Desktop (Electron) │ │ Mobile (Expo / React Native)│ + │ renderer UI (existing) │ │ RN UI (new, MVP scope) │ + │ better-sqlite3 cairn.db │ │ expo-sqlite local DB │ + └────────────┬───────────────┘ └─────────────┬───────────────┘ + │ │ + └────────────── SYNC ───────────────┘ + (changelog exchange + reconcile) +``` + +Both databases run the **same schema + migrations** (so rows are structurally +interchangeable) and both implement the **same sync protocol** from `@cairn/core`. The +desktop's existing `.md` dual-write and file-watcher continue to work unchanged; the sync +engine operates at the DB/changelog layer, above them. + +### Sync transport — DECIDED: synced-folder oplog +Standalone + offline needs a rendezvous point for two DBs to exchange changes. **Decision: +each device writes its oplog (+ note `.md` files) to a cloud-synced folder** (iCloud Drive / +Dropbox / Syncthing); the other device replays the peer's oplog on next launch/foreground. + +- **Never sync `cairn.db` itself** — a binary SQLite file + WAL through opaque cloud merge = + corruption/data loss. Only the append-only oplog files and `.md` files transit the synced folder. +- No backend to operate; keeps Cairn local-first. +- Because oplog files are **append-only and per-device** (`oplog-.ndjson` or chunked), + the cloud sync only ever creates or grows distinct files — it never has to merge the same file + from two writers, which is exactly the case cloud file-sync handles safely. +- The protocol is transport-agnostic (§4), so a LAN P2P or hosted relay path can be added later + behind the same interface without reworking the reconcile logic. + +Rejected alternatives: syncing the binary DB (corruption), LAN-only P2P (fails the offline/ +desktop-closed requirement), hosted relay (needs a server; unnecessary for 2 personal devices). + +--- + +## 4. Sync protocol design (the heart of the work) + +Principles: **offline-first, convergent, delete-safe, skew-safe.** + +> **Scope simplification (2 personal devices, single user):** there is no multi-user contention. +> Conflicts only arise when *the same person* edits the same item on both devices while offline. +> This is rare and low-stakes, so a **last-writer-wins + conflict-copy** policy is sufficient — we +> do NOT need CRDTs or three-way text merge. The protocol below is still fully convergent. + +1. **Hybrid Logical Clock (HLC)** per device. Every mutation stamps an HLC value. HLC gives a + total order that respects causality and tolerates clock skew — replacing today's fragile + wall-clock ISO comparison. +2. **Append-only changelog / oplog** per device: `(entity_type, entity_id, op[insert|update|delete], + hlc, field_values_or_delta, origin_device)`. Populated by a single write-wrapper around all + mutations (the renderer already funnels through `queries.ts` and `db-handlers.ts`, so there + is one choke point to instrument). +3. **Tombstones** for deletes on every syncable table, so deletion propagates and rows don't + resurrect. Reuse existing `archived_at` where present; add `deleted_at` elsewhere. +4. **Sync = exchange oplogs since last-seen HLC watermark per peer, then replay in HLC order.** + Convergent by construction (all peers replay the same ops in the same order). +5. **Conflict resolution policy (per entity type):** + - Scalar fields (title, priority, column, etc.): **last-writer-wins by HLC**. + - **Note body**: text is the high-risk field. Prefer **field-level LWW** for MVP, with a + **conflict copy** (` (conflicted copy, ,