Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
7c688f6
feat: add database reconciliation planner
lirazsiri Jul 28, 2026
777bb19
fix(reconciliation): enforce marker and dependency invariants
lirazsiri Jul 28, 2026
49fb7a3
fix(reconciliation): close validation and resource gaps
lirazsiri Jul 28, 2026
6bb372f
fix(reconciliation): validate physical aliases safely
lirazsiri Jul 28, 2026
12dce92
feat: add transactional database reconciliation apply
lirazsiri Jul 28, 2026
f3c7455
fix(reconciliation): harden apply locking boundaries
lirazsiri Jul 28, 2026
972a026
fix(reconciliation): close locking portability gaps
lirazsiri Jul 28, 2026
063064c
feat(db-sync): add durable snapshot recovery
lirazsiri Jul 28, 2026
063b73a
fix(db-sync): harden persistent snapshot recovery
lirazsiri Jul 28, 2026
c19578c
fix(db-sync): bind snapshot filesystem mutations
lirazsiri Jul 28, 2026
7a2cd4b
fix(db-sync): fail closed on snapshot discovery
lirazsiri Jul 28, 2026
a0cfc27
fix(db-sync): canonicalize private snapshot roots
lirazsiri Jul 29, 2026
3b93bc7
fix(db-sync): avoid persistent cleanup for private snapshots
lirazsiri Jul 29, 2026
f9b02d9
fix(db-sync): prune retained snapshots at zero retention
lirazsiri Jul 29, 2026
ff507e3
feat(db-sync): add standalone reconciliation command
lirazsiri Jul 29, 2026
3fcca50
fix(db-sync): reject repeated singleton options
lirazsiri Jul 29, 2026
0f2c799
feat(db-sync): propagate roster deletion tombstones
lirazsiri Jul 29, 2026
9711b15
test(db-sync): normalize secure fixture modes
lirazsiri Jul 29, 2026
ac3a1d0
feat(db-sync): bootstrap an absent database
lirazsiri Jul 30, 2026
d811e1d
fix(db): prefer later versioned records during sync
lirazsiri Aug 2, 2026
e800563
fix(db): address reconciliation review findings
lirazsiri Aug 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,11 +72,12 @@ skills/ Skill prompt files (brainstorm, wish, work, revi

## CLI Commands

Fourteen top-level commands (run `genie <command> --help` for detail):
Fifteen top-level commands (run `genie <command> --help` for detail):

| Command | Purpose |
|---------|---------|
| `board` | Kanban view derived by query (no stored view state); `--board`, `--wish`, `--json` |
| `db` | Standalone database reconciliation (`genie db sync`) with snapshots, recovery, and rollback |
| `doctor` | Diagnostic checks on the genie installation |
| `hook` | Hook middleware for Claude Code (`genie hook dispatch` runs in-process) |
| `init` | Scaffold per-repo state and reconcile `.mcp.json`, `.warp/.mcp.json`, plus the marker-owned `.codex/config.toml` stable-facade route |
Expand Down
172 changes: 171 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Re-run `genie board` any time for a current snapshot of task state on the kanban
- **Skills** carry the methodology — `brainstorm → design review → wish → plan review → work → implementation review`, authored once for native Claude and Codex surfaces.
- **Documents in git.** Wishes, designs, and brainstorms are plain markdown under `.genie/wishes/<slug>/` and `.genie/brainstorms/<slug>/`; you diff, review, and version them like any other code.
- **One file of state.** Tasks, boards, dependency edges, and wish-group execution state live in a single per-repo SQLite file (`.genie/genie.db`), on Bun's built-in engine.
- **Small.** 14 CLI commands, 4 runtime dependencies (`@inquirer/prompts`, `commander`, `zod`, `nats`) — `nats` initializes only when the omni runner starts. A ~0.9 MB single-file bundle. Bun-powered.
- **Small.** 15 CLI commands, 4 runtime dependencies (`@inquirer/prompts`, `commander`, `zod`, `nats`) — `nats` initializes only when the omni runner starts. A ~0.9 MB single-file bundle. Bun-powered.
- **Warp cockpit (optional).** `genie launch <slug>` turns a wish's ready groups into a Warp window — one pane per group, each in its own git worktree running that group's agent on a kickoff prompt. Emitting the launch config works on any platform; opening it needs Warp (macOS/Linux). Everywhere else the config is still written for you to open by hand.
- **Zero daemons, no Postgres.** Nothing runs in the background between invocations.

Expand All @@ -70,6 +70,7 @@ genie --help
| `genie launch` | Open a Warp cockpit for a wish — one pane per ready group, each in its own worktree |
| `genie board` | Kanban view of task state, derived live by query |
| `genie task` | Inspect and drive task state (SQLite, zero-daemon) |
| `genie db sync` | Reconcile two exact-current Genie databases with snapshots and rollback |
| `genie install` | Finish a verified install and deliver selected integrations; Codex activation is deferred to setup |
| `genie mcp` | Serve read-only Genie task/board state over stdio MCP |
| `genie omni` | Bridge agents to WhatsApp via Omni — remote approvals + inbound one-shots (`serve`, `status`, `inbox`, `handshake`) |
Expand Down Expand Up @@ -163,6 +164,175 @@ Documents live in git; operational state lives in one SQLite file. `work` fans a

All linked worktrees of a repository share one `genie.db`, resolved from the git common directory, so a task created in one worktree is immediately visible in another with no sync step.

### Database reconciliation

Use `genie db sync` when two separate Genie repositories or copied
`.genie/genie.db` files have diverged. The command is noninteractive and takes
explicit database paths; it does not silently discover databases. An explicit
path may name a database that active Genie processes also use.

Choose exactly one mode:

```bash
# Conservative bidirectional reconciliation
genie db sync /path/to/left.db /path/to/right.db

# Directional reconciliation with explicit source authority
genie db sync \
--source /path/to/source.db \
--destination /path/to/destination.db
```

Bidirectional mode unions additions. When the same task or wish-group key
differs, the row with the higher `updated_at` value wins and is copied exactly
to both databases. Equal `updated_at` values with unequal row content remain a
conflict. Hire-roster rows and their deletion tombstones resolve by their
`hired_at` or deletion timestamp, so a later explicit re-hire can supersede a
replicated deletion. Boards and unknown metadata have no trustworthy update
version, so differing rows at the same key remain conflicts.
Directional mode makes the source authoritative for shared mutable task,
board, wish-group, hire-roster, and unknown metadata keys. It writes only the
destination. Both modes preserve destination-only rows: ordinary absence never
means deletion.

When exactly one named database is absent and the other is an exact-current
Genie database, sync bootstraps the absent side from the complete logical image
of the existing side, including committed WAL content. Dry-run reports that
intent without creating the file or sidecars; apply uses the same bounded locks
and refuses unsafe parents, path substitution, or a target that appears first.

An explicit roster unhire (the `roster_unhire` UI-bridge operation) is the
exception. It records a versioned tombstone in the existing metadata table, so
a later sync removes the matching hire-roster row instead of resurrecting it.
This changes neither the SQLite schema nor its `user_version`; reconciliation
reports the explicit removal in its deletion count. Tombstones are durable, but
their timestamp participates in reconciliation: a later re-hire wins over an
older deletion, and a later unhire wins over the replicated live row.

Dependency edges are unioned. Task events and legacy stage-log entries preserve
the maximum occurrence count observed on either applicable side. Independent
byte-identical history additions are indistinguishable without ancestry, so
they can be undercounted rather than guessed or duplicated.

Preview the complete schema validation, conflict detection, and logical plan
without write locks, snapshots, or database writes:

```bash
genie db sync /path/to/left.db /path/to/right.db --dry-run --json
```

Dry-run rejects snapshot, retention, rollback, and busy-timeout options because
they have no read-only effect. Ambiguous positional and directional forms also
fail before opening either database for mutation. Every named option is a
singleton: repeating one is an actionable usage error rather than
last-value-wins behavior. Run
`genie db sync --help` for the two accepted forms and all options.

#### Schema compatibility

Reconciliation accepts the complete current Genie schema only. It compares all
eight user tables, their named columns, foreign keys, indexes, uniqueness,
checks, and `user_version`; the version number alone is not enough. It creates
no table, column, index, trigger, or migration.

An older supported additive shape can still have the current `user_version`.
Open that database once through a normal current Genie command, such as
`genie board`, to add the supported current columns and tables. Then retry
reconciliation. Unknown or extra schema objects remain unsupported and are not
normalized.

#### Snapshots, recovery, and rollback

Every mutating run serializes and validates each destination preimage before it
writes. The generation manifest records its format and operation versions,
generation ID, canonical database identities and roles, preimage and planned
postimage digests, payload hashes, creation time, and recovery state. Snapshot
payloads are recovery inputs; Genie restores their logical rows through live
SQLite transactions and never replaces a live database, WAL, or SHM file.

Bidirectional runs derive a stable pair identity from the sorted canonical
paths. Their default root is:

```text
<directory-of-first-canonical-db>/sync-snapshots/<pair-id>
```

Directional runs are role-sensitive. Their default root is:

```text
<directory-of-canonical-destination>/sync-snapshots
```

Use `--snapshot-root /normalized/absolute/path` to override either default.
Incomplete staging directories are not rollback points. Only a fully published
generation with a complete manifest can be recovered or selected.

Before a new mutation, Genie classifies the newest unresolved generation:

- `converged` means every database already has its recorded postimage.
- `recovered` means every database is back at its preimage, including a
transactionally restored mixed preimage/postimage pair.
- `uncertain` means at least one database matches neither recorded image. Genie
overwrites nothing and requires manual inspection.
- `operational-failure` means recovery could not complete or be classified.

When an apply reports `converged` or `recovered` for a prior generation, rerun
the same command. That invocation performed recovery instead of starting the
new requested mutation.

Rollback is explicit source authority over history. Pass the same database
pair and mode used by the generation:

```bash
genie db sync /path/to/left.db /path/to/right.db \
--rollback <generation-id>
```

Genie validates the manifest and canonical targets, snapshots the arbitrary
current state into a new safety generation, then restores the selected older
logical images. Rollback is reported as `rolled-back`, never as automatic crash
recovery.

The default `--keep-snapshots` value is `3`. Any nonnegative integer is valid.
With `--keep-snapshots 0`, Genie uses a private mode-`0700` OS temporary
directory only for same-process recovery and removes it best-effort on exit.
No generation is published to the normal snapshot root, and finalized
persistent generations for that operation are pruned after success. A later
process therefore cannot recover or roll back the zero-retention operation. A
cleanup failure is reported and returns a nonzero exit even when the logical
apply succeeded.

Mutating runs take canonical-path advisory locks for reconciliation-aware
writers, then SQLite write locks for the databases themselves. These locks let
explicit paths safely name active databases without claiming that unrelated
writers are discovered or refused. `--busy-timeout-ms N` bounds the combined
wait for both lock layers. `N` must be between `0` and `2147483647`. A timeout
returns a bounded operational report; the command does not wait indefinitely.

#### Automation report and exit codes

`--json` emits report version `1`. The stable envelope identifies the operation,
mode, status, hashed database identities, logical and schema digests, per-table
change counts, conflict counts, generation IDs, recovery state, bounded failure
codes, and cleanup failures. It never includes task titles, notes, or other row
content. Human output uses the same statuses and abbreviated identities.

| Exit | Meaning |
|------|---------|
| `0` | The requested dry-run, apply, no-op, same-database check, or rollback completed |
| `1` | Commander rejected command syntax or an unknown option |
| `2` | The mode or option combination is invalid; no mutation started |
| `3` | Conservative reconciliation found conflicts; no mutation started |
| `4` | An operational, lock, rollback, observation, or cleanup failure occurred |
| `5` | Recovery or commit state is uncertain; manual inspection is required |
| `6` | A prior generation was recovered or classified converged; rerun the request |

Crashes can leave a complete unresolved manifest. Automatic recovery changes
data only when every current digest matches that manifest's recorded preimage
or postimage. It never guesses after an unrelated write. A crash before a
complete manifest is published leaves no recovery point, and retention zero
cannot support recovery after process loss.

## Omni (WhatsApp bridge)

`genie omni` wires a running agent to WhatsApp through an [Omni](https://automagik.dev) hub, so you can drive approvals and short tasks from your phone.
Expand Down
17 changes: 14 additions & 3 deletions scripts/release-docs.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -749,6 +749,16 @@ describe('Group E release and documentation contracts', () => {
expect(archiveSmoke).toBeGreaterThan(extract);
});

test('database reconciliation docs describe explicit live paths and both bounded lock layers', () => {
const readme = read('README.md').replace(/\s+/g, ' ');
expect(readme).toContain('An explicit path may name a database that active Genie processes also use.');
expect(readme).toContain('canonical-path advisory locks');
expect(readme).toContain('SQLite write locks');
expect(readme).toContain('bounds the combined wait for both lock layers');
expect(readme).toContain('repeating one is an actionable usage error');
expect(readme).not.toContain('It never discovers or connects a live shared database.');
});
Comment thread
coderabbitai[bot] marked this conversation as resolved.

test('shipped Codex integration doc carries the exit matrix, trailer, lease, and candidate-channel contract', () => {
const doc = read('plugins/genie/references/codex-integration-map.md');
// Exit matrix (per-command 0/1/2) with the busy code.
Expand Down Expand Up @@ -817,9 +827,10 @@ describe('Group E release and documentation contracts', () => {
).toContain('legacy workspace-write grants are forbidden');
});

test('README and contributor command inventories match the 14-command source surface', () => {
test('README and contributor command inventories match the 15-command source surface', () => {
const expected = [
'board',
'db',
'doctor',
'help',
'hook',
Expand All @@ -836,8 +847,8 @@ describe('Group E release and documentation contracts', () => {
];
const readme = read('README.md');
const contributor = read('CLAUDE.md');
expect(readme).toContain('14 CLI commands');
expect(contributor).toContain('Fourteen top-level commands');
expect(readme).toContain('15 CLI commands');
expect(contributor).toContain('Fifteen top-level commands');
const readmeCommands = [...readme.matchAll(/^\| `genie ([a-z-]+)/gm)].map((match) => match[1]).sort();
const contributorCommands = [...contributor.matchAll(/^\| `([a-z-]+)/gm)].map((match) => match[1]).sort();
expect(readmeCommands).toEqual(expected);
Expand Down
18 changes: 9 additions & 9 deletions src/genie-commands/__tests__/update-command-publication.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,15 @@ function buildReleasePayload(
} {
const payload = join(root, 'payload');
for (const directory of ['.agents', '.claude-plugin', 'plugins/genie', 'skills/review', 'templates']) {
mkdirSync(join(payload, directory), { recursive: true });
mkdirSync(join(payload, directory), { recursive: true, mode: 0o755 });
}
writeFileSync(join(payload, '.agents', 'plugin.json'), '{}\n');
writeFileSync(join(payload, '.claude-plugin', 'marketplace.json'), '{}\n');
writeFileSync(join(payload, 'LICENSE'), 'test fixture\n');
writeFileSync(join(payload, 'VERSION'), `${version}\n`);
writeFileSync(join(payload, 'plugins', 'genie', 'plugin.txt'), 'authenticated plugin payload\n');
writeFileSync(join(payload, 'skills', 'review', 'SKILL.md'), '# Review\n');
writeFileSync(join(payload, 'templates', 'template.txt'), 'template\n');
writeFileSync(join(payload, '.agents', 'plugin.json'), '{}\n', { mode: 0o644 });
writeFileSync(join(payload, '.claude-plugin', 'marketplace.json'), '{}\n', { mode: 0o644 });
writeFileSync(join(payload, 'LICENSE'), 'test fixture\n', { mode: 0o644 });
writeFileSync(join(payload, 'VERSION'), `${version}\n`, { mode: 0o644 });
writeFileSync(join(payload, 'plugins', 'genie', 'plugin.txt'), 'authenticated plugin payload\n', { mode: 0o644 });
writeFileSync(join(payload, 'skills', 'review', 'SKILL.md'), '# Review\n', { mode: 0o644 });
writeFileSync(join(payload, 'templates', 'template.txt'), 'template\n', { mode: 0o644 });
Comment on lines +49 to +57

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

node <<'NODE'
const { mkdtempSync, mkdirSync, writeFileSync, statSync, rmSync } = require('node:fs');
const { join } = require('node:path');
const { tmpdir } = require('node:os');

const root = mkdtempSync(join(tmpdir(), 'genie-mode-check-'));
const previous = process.umask(0o077);

try {
  const directory = join(root, 'directory');
  const file = join(directory, 'file');
  mkdirSync(directory, { recursive: true, mode: 0o755 });
  writeFileSync(file, '', { mode: 0o644 });

  console.log({
    directory: (statSync(directory).mode & 0o777).toString(8),
    file: (statSync(file).mode & 0o777).toString(8),
  });
} finally {
  process.umask(previous);
  rmSync(root, { recursive: true, force: true });
}
NODE

Repository: automagik-dev/genie

Length of output: 191


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Files matching test name:"
fd -a 'update-command-publication\.test\.ts$' . || true

file="$(fd 'update-command-publication\.test\.ts$' . | head -n 1 || true)"
if [ -n "${file:-}" ]; then
  echo "---- outline ----"
  ast-grep outline "$file" || true
  echo "---- relevant lines 1-180 ----"
  sed -n '1,180p' "$file" | nl -ba
fi

echo
echo "Search for tarball/archive creation in this file:"
rg -n "tar|archive|createArchive|package|publish|chmod|chmodSync|mkdirSync|writeFileSync|umask" "$file" || true

Repository: automagik-dev/genie

Length of output: 808


🏁 Script executed:

#!/bin/bash
set -euo pipefail

target="src/genie-commands/__tests__/update-command-publication.test.ts"

echo "---- lines 40-120 ----"
sed -n '40,120p' "$target"

echo
echo "---- lines 120-260 ----"
sed -n '120,260p' "$target"

echo
echo "---- lines 260-420 ----"
sed -n '260,420p' "$target"

echo
echo "---- search archive/tar/chmod/umask references ----"
rg -n "tar|archive|package|publish|chmod|chmodSync|mkdirSync|writeFileSync|umask|sha256|generate|create|Release" "$target" || true

Repository: automagik-dev/genie

Length of output: 7818


Normalize release fixture permissions before archiving.

mkdirSync() and writeFileSync() only apply mode as the per-path umask, so restrictive environments create 0700/ 0600 entries. Since this fixture is archived with tar, set directory permissions after creation and file permissions after writing.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/genie-commands/__tests__/update-command-publication.test.ts` around lines
49 - 57, Update the fixture setup around the mkdirSync and writeFileSync calls
to explicitly normalize permissions after creation: apply 0755 to every created
directory and 0644 to every written file before the archive is generated.
Preserve the existing fixture paths and contents while ensuring restrictive
umasks cannot change the archived entry permissions.

writeExecutable(
join(payload, 'genie'),
`#!/bin/sh\nif [ "\${1:-}" = "--version" ]; then printf 'genie ${version}\\n'; exit 0; fi\nexit 0\n`,
Expand All @@ -77,7 +77,7 @@ describe('updateCommand publication boundary', () => {
const bin = join(genieHome, 'bin');
const fakeBin = join(root, 'fake-bin');
const fixture = join(root, 'fixture');
mkdirSync(bin, { recursive: true });
mkdirSync(bin, { recursive: true, mode: 0o755 });
mkdirSync(fakeBin);
mkdirSync(fixture);

Expand Down
3 changes: 3 additions & 0 deletions src/genie-commands/__tests__/update.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2455,6 +2455,9 @@ describe('manual post-update convergence (2026-07-11 cascade regression)', () =>
const result = runManualUpdateConvergence({
expectedVersion: '5.260711.3',
bundleRoot: '/tmp/verified-bundle',
// Explicit selection: the default reads the host's persisted integration
// consent, so omitting it makes the test depend on machine state (#2732).
selection: 'all',
runSync: () => calls.push('parent-safe-sync'),
refreshPlugins: (options) => {
calls.push(`parent-plugin-refresh:${options.expectedVersion}:${options.selection}`);
Expand Down
2 changes: 1 addition & 1 deletion src/genie-commands/local-delivery-repair.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ function isolatedEnv(root: string, overrides: Record<string, string> = {}): Reco
const genieHome = join(root, 'genie-home');
const codexHome = join(root, 'codex-home');
const temp = join(root, 'tmp');
for (const path of [home, genieHome, codexHome, temp]) mkdirSync(path, { recursive: true });
for (const path of [home, genieHome, codexHome, temp]) mkdirSync(path, { recursive: true, mode: 0o700 });
return {
...env,
HOME: home,
Expand Down
2 changes: 2 additions & 0 deletions src/genie.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import { registerMcpCommand } from './term-commands/mcp.js';
import { registerOmniCommands } from './term-commands/omni.js';
import { registerUiBridgeCommand } from './term-commands/ui-bridge.js';
import { registerV5BoardCommands } from './term-commands/v5-board.js';
import { registerV5DatabaseSyncCommand } from './term-commands/v5-db-sync.js';
import { registerV5TaskCommands } from './term-commands/v5-task.js';

const program = new Command();
Expand Down Expand Up @@ -191,6 +192,7 @@ registerMcpCommand(program);
registerUiBridgeCommand(program);
registerV5TaskCommands(program);
registerV5BoardCommands(program);
registerV5DatabaseSyncCommand(program);
registerIdeaCommand(program);
registerOmniCommands(program);

Expand Down
Loading
Loading