Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ pnpm db:migrate:remote # Apply to production D1 (requires auth)
- **Cowork `result` billing**: each `type: "result"` record is one completed host-loop run's final bill — `usage`, `modelUsage`, `total_cost_usd` and `duration_ms` are per-run, not cumulative. Session totals are the sum over UUID-deduplicated results; never mix them with assistant `message.usage` snapshots (those are partial streams and undercount output badly), and never add `usage` to `modelUsage` or `total_cost_usd` to `modelUsage.*.costUSD`. Audits with no `result` records fall back to assistant snapshots plus the timestamp-gap duration estimate. `system/api_retry` maps to `apiErrors` with attempt metadata only — the raw error text is never stored.
- **Provider-reported cost**: `ProviderParseResult.reportedCostUsd` wins over the local pricing table in `transform.ts`, which is what makes cost available for model generations `pricing.ts` does not know.
- **opencode**: Sessions live in a SQLite DB at `~/.local/share/opencode/opencode.db` (`%LOCALAPPDATA%\\opencode` on Windows, `OPENCODE_DATA` env var wins). Tables: `session`/`message`/`part`; role/user and tool payloads are JSON in `message.data`/`part.data`. Discovery writes `<dbPath>#session:<id>` marker paths. Parse is SQLite-backed (`parseSessionFromDb`), so background scan uses a lightweight path from discovery-computed stats (`buildLightweightOpencodeScanResult` in `scanner.ts`) to avoid opening the DB per session. `sql.js` named-param binds need their `:`-prefix in the key — this provider uses positional `?` params instead. See `packages/provider-opencode/`.
- **Hermes**: Sessions live in SQLite at `~/.hermes/state.db` (FTS5-backed). Tables: `sessions` (token/cost/git columns maintained by Hermes) + `messages` (OpenAI-style: assistant rows carry `tool_calls` JSON + `reasoning`; tool rows carry `tool_name` + `tool_call_id` + result content). Discovery writes `<dbPath>#session:<id>` marker paths and reads `~/.hermes/.update_check` for the version. Context compaction is recorded two ways: `compacted=1` rows (pre-compaction history, kept in full) and a user row prefixed `[CONTEXT COMPACTION` (→ `subtype: "compaction-summary"`, mirroring claude-code). Parse is SQLite-backed, so background scan uses `buildLightweightHermesScanResult` in `scanner.ts`. Tool names map to the viewer vocabulary in `tool-mapping.ts`. See `packages/provider-hermes/`.
- **Hermes**: Sessions live in SQLite at `~/.hermes/state.db` (FTS5-backed), plus one DB per named profile at `~/.hermes/profiles/<name>/state.db`. Discovery scans all of them (`hermesDbPaths()` in `packages/provider-hermes/src/hermes/sqlite.ts`), dedups by session id, and writes `<dbPath>#session:<id>` marker paths; parse resolves the right DB from the marker before falling back to a scan. `hermesRootDir()` mirrors Hermes's `get_default_hermes_root()`: `HERMES_HOME` inside `~/.hermes` is profile mode (root stays `~/.hermes`); outside it, that path itself is the root. Caveat: sql.js reads raw file bytes and cannot replay un-checkpointed `-wal` frames, so sessions still sitting in the WAL (long-running gateway) are invisible until Hermes checkpoints. Tables: `sessions` (token/cost/git columns maintained by Hermes) + `messages` (OpenAI-style: assistant rows carry `tool_calls` JSON + `reasoning`; tool rows carry `tool_name` + `tool_call_id` + result content). Discovery writes `<dbPath>#session:<id>` marker paths and reads `~/.hermes/.update_check` for the version. Context compaction is recorded two ways: `compacted=1` rows (pre-compaction history, kept in full) and a user row prefixed `[CONTEXT COMPACTION` (→ `subtype: "compaction-summary"`, mirroring claude-code). Parse is SQLite-backed, so background scan uses `buildLightweightHermesScanResult` in `scanner.ts`. Tool names map to the viewer vocabulary in `tool-mapping.ts`. See `packages/provider-hermes/`.
- **Skip `progress` lines**: These are subagent streaming artifacts in JSONL.
- **sql.js (WASM)**: Used instead of native SQLite bindings for portability — no C++ compiler needed.
- **Session discovery cache**: CLI picker + local dashboard use file cache at `~/.vibe-replay/cache/*.json` (stale-while-refresh UX). Cache validity is tied to CLI release version (`CLI_VERSION`) plus envelope version, so caches auto-invalidate across releases. Keep cache writes best-effort and never block generation/parsing on cache failures.
Expand Down
39 changes: 29 additions & 10 deletions packages/provider-hermes/src/hermes/discover.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import type { Database } from "sql.js";
import { cleanPromptText } from "@vibe-replay/provider-core/clean-prompt";
import type { SessionInfo } from "@vibe-replay/provider-contract";
import { shortenPath } from "@vibe-replay/provider-core/utils";
import { hermesDataDir, hermesDbPath, openHermesDb } from "./sqlite.js";
import { hermesDataDir, hermesDbPath, openAllHermesDbs } from "./sqlite.js";

export const HERMES_PROVIDER = "hermes";

Expand Down Expand Up @@ -47,17 +47,34 @@ function rowValues(db: Database, sql: string, params: Record<string, any> = {}):
}

export async function discoverHermesSessions(): Promise<SessionInfo[]> {
const opened = await openHermesDb();
if (!opened) return [];
const { db } = opened;
const all = await openAllHermesDbs();
if (all.length === 0) return [];
const sessions: SessionInfo[] = [];
const seen = new Set<string>();
try {
return listSessionsFromDb(db);
for (const { db, dbPath } of all) {
const fromDb = listSessionsFromDb(db, dbPath);
for (const s of fromDb) {
if (seen.has(s.sessionId)) continue;
seen.add(s.sessionId);
sessions.push(s);
}
}
} finally {
db.close();
for (const { db } of all) {
try {
db.close();
} catch {
// ignore
}
}
}
// Deterministic ordering: newest last_activity first across profiles.
sessions.sort((a, b) => b.timestamp.localeCompare(a.timestamp));
return sessions;
}

export function listSessionsFromDb(db: Database): SessionInfo[] {
export function listSessionsFromDb(db: Database, dbPathOverride?: string): SessionInfo[] {
const rows = rowValues(
db,
`
Expand All @@ -77,11 +94,12 @@ export function listSessionsFromDb(db: Database): SessionInfo[] {
const statsBySession = buildSessionStats(db, rows);

const version = hermesVersion();
const fallbackDbPath = dbPathOverride ?? hermesDbPath();
const sessions: SessionInfo[] = [];
for (const row of rows) {
const stats = statsBySession.get(row.id);
if (!stats) continue;
const info = sessionInfoFromRow(row, stats, version);
const info = sessionInfoFromRow(row, stats, version, fallbackDbPath);
if (info) sessions.push(info);
}
return sessions;
Expand Down Expand Up @@ -180,14 +198,15 @@ function sessionInfoFromRow(
row: HermesSessionRow,
stats: SessionStats,
version: string,
dbPath?: string,
): SessionInfo | null {
if (!row.id) return null;

const firstPrompt = cleanPromptText(stats.firstPrompt);
if (!firstPrompt) return null;

const dbPath = hermesDbPath();
const markerPath = `${dbPath}#session:${row.id}`;
const resolvedDbPath = dbPath ?? hermesDbPath();
const markerPath = `${resolvedDbPath}#session:${row.id}`;
const lastActivity = row.last_activity_at ?? row.ended_at ?? row.started_at;

return {
Expand Down
83 changes: 74 additions & 9 deletions packages/provider-hermes/src/hermes/parser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,13 @@ import type {
TokenUsage,
} from "@vibe-replay/provider-contract";
import { addParseWarning, compactWarningSample } from "@vibe-replay/provider-contract/warnings";
import { openHermesDb, hermesDataDir, hermesDbPath } from "./sqlite.js";
import {
hermesDataDir,
hermesDbPath,
hermesDbPaths,
openAllHermesDbs,
openHermesDb,
} from "./sqlite.js";
import { mapHermesToolArgs, mapHermesToolName } from "./tool-mapping.js";

interface HermesMessageRow {
Expand Down Expand Up @@ -91,18 +97,76 @@ export async function parseHermesSession(
);
}

const opened = await openHermesDb();
if (!opened) {
throw new Error(`Hermes database not found at ${hermesDbPath()}`);
// Prefer the DB hinted by the marker path (fast path for the common case).
const hinted = hintedDbPath(paths, sessionInfo);
if (hinted) {
const opened = await openHermesDb(hinted);
if (opened) {
try {
const row = firstValue(opened.db, "SELECT id FROM sessions WHERE id = ?", {
sid: sessionId,
});
if (row) return parseSessionFromDb(opened.db, sessionId, sessionInfo, opened.dbPath);
} finally {
opened.db.close();
}
}
}

const all = await openAllHermesDbs();
if (all.length === 0) {
throw new Error(
`Hermes database not found (searched: ${hermesDbPaths().join(", ") || hermesDbPath()})`,
);
}
const { db } = opened;
// Find the winning DB while handles are live, close everything, then re-open
// just the winner fresh for parsing — keeps WASM handle ownership simple.
let winnerPath: string | undefined;
for (const entry of all) {
try {
const row = firstValue(entry.db, "SELECT id FROM sessions WHERE id = ?", { sid: sessionId });
if (row) {
winnerPath = entry.dbPath;
break;
}
} catch {
// ignore per-DB probe errors
}
}
for (const { db } of all) {
try {
db.close();
} catch {
// ignore
}
}
if (!winnerPath) {
throw new Error(`Hermes session '${sessionId}' not found in any known database`);
}
const opened = await openHermesDb(winnerPath);
if (!opened) throw new Error(`Hermes session '${sessionId}' not found in any known database`);
try {
return parseSessionFromDb(db, sessionId, sessionInfo);
return parseSessionFromDb(opened.db, sessionId, sessionInfo, winnerPath);
} finally {
db.close();
opened.db.close();
}
}

function hintedDbPath(paths: string[], sessionInfo?: SessionInfo): string | undefined {
for (const p of paths) {
const idx = p.indexOf("#session:");
if (idx >= 0) {
const dbPath = p.slice(0, idx);
if (dbPath) return dbPath;
}
}
const fp = (sessionInfo as unknown as { filePath?: string })?.filePath;
if (typeof fp === "string" && fp.includes("#session:")) {
return fp.split("#session:")[0];
}
return undefined;
}

/**
* Extract the Hermes session id from the filePath markers written by
* discovery (`<dbPath>#session:<id>`), or from sessionInfo, or from a raw
Expand All @@ -129,6 +193,7 @@ export function parseSessionFromDb(
db: Database,
sessionId: string,
sessionInfo?: SessionInfo,
sourceDbPath?: string,
): ProviderParseResult {
const session = firstValue(db, `SELECT * FROM sessions WHERE id = ?`, {
sid: sessionId,
Expand Down Expand Up @@ -258,8 +323,8 @@ export function parseSessionFromDb(

const defaultSource: DataSourceInfo = {
primary: "sqlite",
sources: [hermesDbPath()],
notes: ["Discovered from the Hermes SQLite database (~/.hermes/state.db)."],
sources: [sourceDbPath ?? hermesDbPath()],
notes: ["Discovered from the Hermes SQLite database."],
};

return {
Expand Down
77 changes: 76 additions & 1 deletion packages/provider-hermes/src/hermes/sqlite.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
/// <reference path="../sql-js.d.ts" />
import { existsSync, readdirSync, realpathSync } from "node:fs";
import { readFile } from "node:fs/promises";
import { homedir } from "node:os";
import { join } from "node:path";
import { basename, dirname, isAbsolute, join, relative, resolve } from "node:path";
import type { Database, SqlJsStatic } from "sql.js";
import type { Dirent } from "node:fs";

export const HERMES_DIRNAME = ".hermes";
export const HERMES_DB_FILENAME = "state.db";
Expand All @@ -19,6 +21,64 @@ export function hermesDbPath(): string {
return join(hermesDataDir(), HERMES_DB_FILENAME);
}

function resolveExisting(p: string): string {
try {
return realpathSync(p);
} catch {
return p;
}
}

/**
* Root directory that holds the default DB and the named profiles — mirrors
* Hermes's own `get_default_hermes_root()`:
*
* - No `HERMES_HOME`: root is `~/.hermes`.
* - `HERMES_HOME` inside `~/.hermes` (profile mode, e.g. a path under
* `~/.hermes/profiles/`): root stays `~/.hermes` so all profiles are visible.
* - `HERMES_HOME` outside `~/.hermes` (Docker/custom deployment): root is
* `HERMES_HOME` itself — unless it points at `<root>/profiles/<name>`, in
* which case root is that grandparent.
*/
export function hermesRootDir(): string {
const envHome = process.env.HERMES_HOME;
const nativeHome = join(homedir(), HERMES_DIRNAME);
if (!envHome) return nativeHome;
const envPath = resolveExisting(resolve(envHome));
const rel = relative(resolveExisting(nativeHome), envPath);
if (rel === "" || (!rel.startsWith("..") && !isAbsolute(rel))) {
return nativeHome;
}
if (basename(dirname(envPath)) === "profiles") {
return dirname(dirname(envPath));
}
return envPath;
}

/**
* All `state.db` paths that belong to this Hermes install: the default home
* plus every named profile's DB (`<root>/profiles/<name>/state.db`). Missing
* locations are skipped so partial installs still work.
*/
export function hermesDbPaths(): string[] {
const out: string[] = [];
const defaultDb = join(hermesRootDir(), HERMES_DB_FILENAME);
if (existsSync(defaultDb)) out.push(defaultDb);
const profilesDir = join(hermesRootDir(), "profiles");
let entries: Dirent[] = [];
try {
entries = readdirSync(profilesDir, { withFileTypes: true });
} catch {
// no profiles dir — default DB only
}
for (const entry of entries) {
if (!entry.isDirectory()) continue;
const p = join(profilesDir, entry.name, HERMES_DB_FILENAME);
if (existsSync(p)) out.push(p);
}
return out;
}

export function createRetryableInit<T>(factory: () => Promise<T>): () => Promise<T> {
let promise: Promise<T> | null = null;
return async () => {
Expand Down Expand Up @@ -67,6 +127,21 @@ export async function openHermesDb(
}
}

/**
* Open each known Hermes DB (default + profiles). Useful for discovery and
* for parsing a session that could live in any profile.
*/
export async function openAllHermesDbs(): Promise<Array<{ db: Database; dbPath: string }>> {
const paths = hermesDbPaths();
if (paths.length === 0) return [];
const out: Array<{ db: Database; dbPath: string }> = [];
for (const p of paths) {
const opened = await openHermesDb(p);
if (opened) out.push(opened);
}
return out;
}

/** True when the session id looks like a Hermes session id (`YYYYMMDD_HHMMSS_...`). */
export function isHermesSessionId(value: string): boolean {
return /^\d{8}_\d{6}_/.test(value) || value.startsWith("session_");
Expand Down
14 changes: 14 additions & 0 deletions packages/provider-hermes/src/hermes/tool-mapping.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,20 @@ export function mapHermesToolName(name: string): string {
execute_code: "ExecuteCode",
session_search: "SessionSearch",
text_to_speech: "TextToSpeech",
// Browser + cron + memory tools: no special scene handling in the
// transform yet, but CamelCase ids read better than raw snake_case in
// tool lists and labels.
browser_navigate: "BrowserNavigate",
browser_click: "BrowserClick",
browser_type: "BrowserType",
browser_snapshot: "BrowserSnapshot",
browser_console: "BrowserConsole",
browser_scroll: "BrowserScroll",
browser_press: "BrowserPress",
browser_get_images: "BrowserImages",
cronjob: "Cron",
memory: "Memory",
codex: "Codex",
};
return mapping[name] || name;
}
Expand Down
2 changes: 2 additions & 0 deletions packages/provider-hermes/src/sql-js.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ declare module "sql.js" {
exec(sql: string, params?: any[]): QueryExecResult[];
prepare(sql: string): Statement;
run(sql: string, params?: any[] | Record<string, any>): Database;
/** Dump the database contents as SQLite file bytes (e.g. for persisting to disk). */
export(): Uint8Array;
close(): void;
}

Expand Down
Loading