Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
4df1fee
feat: add CuSessionFinalized IPC message and QA metadata to CuSession…
Jasonnnz Feb 23, 2026
f04fe3d
feat: implement CU session finalization handler with QA metadata plum…
Jasonnnz Feb 23, 2026
99d922d
fix: update IPCCuSessionCreate init and add Ms suffix to INT_PATTERNS…
Jasonnnz Feb 23, 2026
7a02a9c
feat: add file-backed attachment storage and content streaming endpoi…
Jasonnnz Feb 23, 2026
6de8215
fix: preserve CU metadata until finalization, fix latencyMs type in I…
Jasonnnz Feb 23, 2026
7b4b32d
fix: preserve new columns in legacy migration and clamp range ends pe…
Jasonnnz Feb 23, 2026
e02dbe8
feat: add macOS screen recorder, CU session QA integration, and video…
Jasonnnz Feb 23, 2026
f955aa4
feat: add retention cleanup, config, tests, and architecture docs for…
Jasonnnz Feb 23, 2026
00041b8
fix: finalize QA recording before abort and use defer for recorder cl…
Jasonnnz Feb 23, 2026
57595f5
fix: clean up cuSessionMetadata on socket disconnect and add NOT NULL…
Jasonnnz Feb 23, 2026
2a89a60
fix: clamp cleanup interval to setInterval-safe maximum (#6925)
Jasonnnz Feb 23, 2026
ba6b6b9
fix: defer abort message in cancel() for QA mode until after finaliza…
Jasonnnz Feb 23, 2026
7b3874e
fix: keep session in socketToCuSession until socket close to prevent …
Jasonnnz Feb 23, 2026
2cc5a0b
fix: add safety-net abort in cancel() for QA mode with delay (#6932)
Jasonnnz Feb 23, 2026
4d57a54
fix: remove stale socketToCuSession entry on CU session replacement (…
Jasonnnz Feb 23, 2026
fd67967
fix: cancel safety-net timer when run() reaches finalization (#6934)
Jasonnnz Feb 23, 2026
c648898
feat: QA intent detection + routing metadata + epoch fix (#6941)
Jasonnnz Feb 23, 2026
93a3ee4
feat: create file-backed attachment on finalize + video view /content…
Jasonnnz Feb 23, 2026
72ee8bb
feat: wire QA mode from task_routed to ComputerUseSession (#6945)
Jasonnnz Feb 23, 2026
6a2049c
feat: wire reportToSessionId for direct QA + configurable retention d…
Jasonnnz Feb 23, 2026
0db6e03
docs: fix ARCHITECTURE.md QA recording data flow accuracy (#6952)
Jasonnnz Feb 23, 2026
4a1fe4f
fix: recording fallback tracks all untracked recordings, not just orp…
Jasonnnz Feb 23, 2026
09fea78
feat: wire captureScope and includeAudio from daemon config to client…
Jasonnnz Feb 23, 2026
8b86cf0
fix: apply captureScope config to resolve window/display ID for recor…
Jasonnnz Feb 23, 2026
251c116
fix: exclude own PID from window capture to avoid recording Vellum ov…
Jasonnnz Feb 23, 2026
7811e1c
fix: narrow QA intent regex + disarm safety-net via Task.isCancelled …
Jasonnnz Feb 23, 2026
f414d5b
feat: show toast when CU ends without success
Feb 23, 2026
32d1f18
feat: target-app scoping, QA routing, and CU overlay permissions
Feb 23, 2026
ac326ea
feat: strict target-app guard + hardened openApp resolution (#7281)
Jasonnnz Feb 23, 2026
4ae4f4c
feat: propagate target app through IPC + frontmost-app runtime guard …
Jasonnnz Feb 23, 2026
b338968
feat: proactive daemon-side auto-approve for CU sessions (#7288)
Jasonnnz Feb 23, 2026
ed4859e
fix: preserve pending prompt on retro-approve send failure (#7290)
Jasonnnz Feb 23, 2026
0b04313
fix: handle legacy denied screen recording on first request (#7291)
Jasonnnz Feb 23, 2026
0d40ca5
fix: tighten cross-app escape hatch + guard empty fuzzy match (#7292)
Jasonnnz Feb 23, 2026
a4be32c
fix: frontmost guard consecutive-block termination + CuSessionCreate …
Jasonnnz Feb 23, 2026
7a4623a
fix: QA auto-approve from session start + recorder salvage + toast UX…
Jasonnnz Feb 24, 2026
af41f7b
feat: IPC contract + config flag for recording enforcement (#7383)
Jasonnnz Feb 24, 2026
23d945a
feat: daemon action gate blocks destructive tools before recording st…
Jasonnnz Feb 24, 2026
5f02309
feat: thread QA latch + deterministic requiresRecording routing (#7387)
Jasonnnz Feb 24, 2026
6354473
feat: fail-closed recorder handshake with CuRecordingStatus IPC (#7388)
Jasonnnz Feb 24, 2026
6bddc71
feat: recording status indicator + expandable error text in CU overla…
Jasonnnz Feb 24, 2026
5f480ee
fix: canonicalize app name variants in cross-app detection (#7409)
Jasonnnz Feb 24, 2026
d52b122
fix: avoid double prompt on fresh screen recording request (#7411)
Jasonnnz Feb 24, 2026
6475fb3
fix: frontmost guard sends abort + resets counter on non-guard blocks…
Jasonnnz Feb 24, 2026
a72e75b
merge: incorporate latest main into feature/qa-video-automation
Feb 24, 2026
9390bdb
feat: enable enforceStartBeforeActions by default
Feb 24, 2026
e9fddf2
fix: set QA latch on user_message path for CU escalation (#7424)
Jasonnnz Feb 24, 2026
b3c8e07
feat: improve recording gate observability and user feedback (#7425)
Jasonnnz Feb 24, 2026
4311412
feat: generalize target-app-hints to detect common macOS apps (#7428)
Jasonnnz Feb 24, 2026
8bc2a6a
fix: prevent Vellum from stealing focus during external-app QA sessio…
Jasonnnz Feb 24, 2026
21ced14
feat: thread app_bundle_id through open_app tool end-to-end (#7434)
Jasonnnz Feb 24, 2026
89b9145
fix: harden openApp resolution with expanded aliases and mdfind fallb…
Jasonnnz Feb 24, 2026
4c9d27c
fix: mdfind termination handler ordering and single-quote escaping (#…
Jasonnnz Feb 24, 2026
dd1e54f
fix: narrow target-app-hints to avoid false-positive matches (#7457)
Jasonnnz Feb 24, 2026
c98832f
fix: prevent double QA finalization on frontmost guard abort (#7458)
Jasonnnz Feb 24, 2026
44f307d
fix: defer QA latch updates until message is accepted (#7459)
Jasonnnz Feb 24, 2026
6705478
fix: keep Gmail distinct from Mail in APP_CANONICAL_MAP (#7460)
Jasonnnz Feb 24, 2026
cb58b99
fix: restrict bundle ID injection to target app matches only (#7461)
Jasonnnz Feb 24, 2026
9a9f092
fix: pass qaMode explicitly to ComputerUseSession constructor (#7462)
Jasonnnz Feb 24, 2026
1b9bb3f
fix: external target detection fallback and name-based gating (#7463)
Jasonnnz Feb 24, 2026
c11b38c
fix: use double-quoted Spotlight query for mdfind app names (#7464)
Jasonnnz Feb 24, 2026
c8a3c79
fix: use defer for didFinalizeQARecording flag in all return paths (#…
Jasonnnz Feb 24, 2026
6666a83
fix: add name-based fallback to Session.swift isExternalTarget check …
Jasonnnz Feb 24, 2026
990c616
fix: escape backslashes in mdfind Spotlight query sanitization (#7511)
Jasonnnz Feb 24, 2026
01c86e8
fix: escape backslashes in mdfind Spotlight query sanitization (#7515)
Jasonnnz Feb 24, 2026
6e39dce
feat: strict visual QA mode with focus-lock and recording validation
Feb 24, 2026
4f0ed40
Merge remote-tracking branch 'origin/main' into feature/qa-video-auto…
Feb 24, 2026
458446d
feat(macos): add AX-first action targeting and focus reliability
Feb 24, 2026
ea26e3e
feat(skills): add screen-recording awareness skill (#8011)
Jasonnnz Feb 24, 2026
3d1527e
feat(skills): add qa-testing orchestration skill (#8015)
Jasonnnz Feb 24, 2026
1b0d9d2
fix(macos): decouple ScreenRecorder instantiation from qaMode (#8018)
Jasonnnz Feb 24, 2026
3026881
feat: allow requiresRecording to be set independently of QA intent (#…
Jasonnnz Feb 24, 2026
9151823
fix: always forward RUNTIME_HTTP_PORT to daemon for video playback
Feb 24, 2026
6c53b25
feat: standalone screen recording and self-targeted CU sessions
Feb 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 49 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4321,6 +4321,53 @@ Keep-alive heartbeats (every 30 s by default):

---

## QA Recording — Automated Video Capture and Retention

### QA Recording Data Flow

```
User asks to test → QA intent detection (regex/keyword) → CU session created with qaMode
→ reportToSessionId set if conversationId available (chat context) or sourceSessionId (escalation)
→ macOS ScreenRecorder starts → CU action loop executes
→ Session terminates → ScreenRecorder stops → .mp4 saved to ~/Library/Application Support/vellum-assistant/recordings/
→ cu_session_finalized sent to daemon with recording metadata
→ Daemon handler creates file-backed attachment (always, for cleanup tracking)
→ If reportToSessionId present: also injects assistant message + attachment into source chat
→ Client loads video from GET /v1/attachments/:id/content (with Range support)
→ Video playable inline + draggable to Finder
→ Retention cleanup removes expired recordings after configurable period (qaRecording.defaultRetentionDays)
```

### File-Backed Attachment Storage

The attachments table supports two storage kinds:

| `storageKind` | Data location | Use case |
|---------------|---------------|----------|
| `inline_base64` | `dataBase64` column in SQLite | Small attachments (images, documents, up to 20 MB) |
| `file` | On-disk file referenced by `filePath` column | Large files (QA recordings, videos) |

File-backed attachments store only metadata in SQLite (filename, MIME type, size, SHA-256 hash, expiry timestamp). Binary content is served via `GET /v1/attachments/:id/content` with HTTP Range header support for streaming video playback.

### Retention Cleanup

A periodic cleanup worker (`recording-cleanup.ts`) runs on a configurable interval (default: every 6 hours, set via `qaRecording.cleanupIntervalMs`). It also runs one pass on daemon startup to catch recordings that expired while the daemon was offline.

The cleanup pass:
1. Queries `getExpiredFileAttachments()` for file-backed attachments where `expiresAt < now`
2. Deletes the underlying file from disk via `fs.unlinkSync`
3. Removes the DB row via `deleteFileBackedAttachment(id)`
4. Logs a summary of cleaned-up recordings and freed disk space

| Key files | Purpose |
|-----------|---------|
| `assistant/src/daemon/recording-cleanup.ts` | Cleanup worker (start/stop/runPass) |
| `assistant/src/memory/attachments-store.ts` | `createFileBackedAttachment`, `getExpiredFileAttachments`, `deleteFileBackedAttachment` |
| `assistant/src/config/schema.ts` | `QaRecordingConfigSchema` (retention days, cleanup interval) |
| `assistant/src/daemon/lifecycle.ts` | Wires cleanup worker start/stop into daemon init/shutdown |

---

## Storage Summary

| What | Where | Format | ORM/Driver | Retention |
Expand All @@ -4338,7 +4385,8 @@ Keep-alive heartbeats (every 30 s by default):
| Entity graph (entities/relations/item links) | `~/.vellum/workspace/data/db/assistant.db` | SQLite | Drizzle ORM | Permanent, deduped by unique relation edge |
| Embeddings | `~/.vellum/workspace/data/db/assistant.db` | JSON float arrays | Drizzle ORM | Permanent |
| Async job queue | `~/.vellum/workspace/data/db/assistant.db` | SQLite | Drizzle ORM | Completed jobs persist |
| Attachments | `~/.vellum/workspace/data/db/assistant.db` | Base64 in SQLite | Drizzle ORM | Permanent |
| Attachments (inline) | `~/.vellum/workspace/data/db/assistant.db` | Base64 in SQLite | Drizzle ORM | Permanent |
| Attachments (file-backed) | `~/Library/Application Support/vellum-assistant/recordings/` + metadata in SQLite | Binary on disk, metadata in SQLite | Drizzle ORM + fs | Configurable (`qaRecording.defaultRetentionDays`, default 7 days) |
| Sandbox filesystem | `~/.vellum/workspace` | Real filesystem tree | Node FS APIs | Persistent across sessions |
| Tool permission rules | `~/.vellum/protected/trust.json` | JSON | File I/O | Permanent |
| Web users & assistants | PostgreSQL | Relational | Drizzle ORM (pg) | Permanent |
Expand Down
Loading