Bound activity payload bytes, and reclaim what the uncapped ones already wrote

[tusharbhardwaj-bk]

Why

bkt3's state.sqlite grew 3.30 → 3.61 GB in two days and was the largest untouched contributor to disk pressure on a host that fell to 21 GB free during this work.

Two measurements redirected the obvious fix:

Measurement	Value	Consequence
Events older than 30 days	3,912 rows / 9 MB	age-based retention is worthless — the DB is only ~30 days deep
freelist_count	0	VACUUM alone would have reclaimed nothing

The bloat is unbounded tool output. OrchestrationThreadActivity.payload is Schema.Unknown — whatever the provider adapter produced, verbatim — and it is stored twice: in the append-only event log, and again in projection_thread_activities. Bytes by JSON path across rows over 20 KB:

241 MB  $.data.item.aggregatedOutput           command stdout/stderr
219 MB  $.data.item.result.content[].text      MCP tool result text
 24 MB  ...thread.activities[]...aggregatedOutput

That third path is the pathological one: T3's own MCP tools return thread activities, so a t3_get_session call embeds other activities' captured output inside its own result, which is stored as a new activity. The database was feeding on itself, and every round trip was re-logged through the WAL.

What this does

activityPayloadCap.ts caps strings at 32 KiB, arrays at 1,000 items, depth at 24 — applied at decider.ts, where the event is constructed. Because the projection is derived from the event, one cap covers both tables.

It truncates with a self-describing marker rather than dropping fields: unlike a never-rendered blob, this content is shown in the timeline, and the head of a command's output is the part anyone reads. Payloads already under the caps return the same reference, so the ~98% common case neither copies nor re-serializes.

The caps are idempotent by construction — the marker is charged against the budget, so a capped value lands at the cap and never above it. The backfill re-reads rows it may already have rewritten, and a non-idempotent cap would eat real output on every pass.

compact-activity-payloads.ts rewrites rows written before the cap existed. It imports the server's own capActivityPayload, so backfill and runtime cannot drift. It rewrites in place — no row, no sequence, and no thread history is deleted, so the projection bootstrap replay (from the persisted cursor) and the WebSocket reconnect replay are structurally untouched.

Verification

Run against real production payloads from a pre-change backup:

  itemType             before      after   ratio  valid-JSON  idempotent
  image_view            1658KB       35KB      2%        true        true
  image_view            1558KB       35KB      2%        true        true
  longest string after capping: 32713 (cap 32768) -> PASS

Already applied to both live databases:

	expbkt3	bkt3
Reclaimed	29 MB	696 MB
Rows rewritten	338	6,960 (0.9% of rows)
Integrity / cursor == maxSeq	ok	ok
Restarts	0	0

Growth has stopped on bkt3: the file has been flat at 3,491 MB for 6.75 h while free pages were consumed (690 → 654 MB) rather than the file being extended. Live bytes are down to 2,837 MB. Zero errors on either instance over 6 h. The cap has run on expbkt3 since 7066da17.

Known limits

• The cap has not yet been observed firing on live traffic — expbkt3 was idle for the 6 h after deploy. It is verified by unit tests and against real production payloads, but bkt3's load is what will actually exercise it.
• The file does not shrink until a VACUUM, which needs the service stopped. Growth stopping was the actual problem; the ~700 MB of file space is a separate scheduled step.
• A lower cap reclaims more: 16 KB → ~816 MB, 8 KB → ~1,014 MB, against 3,477 rows touched at 32 KB vs 16,751 at 8 KB.

Operator note

The first production backfill run used 100 rows per transaction and held SQLite's single write lock long enough that two runtime event appends were dropped (thread.token-usage.updated, task.progress — telemetry only; no thread content, message, or user action). Defaults are now 25 rows / 250 ms, with the reason recorded in the file header. Our own busy timeout does not help there: it governs how long we wait, not how long the server waits on us.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	—	11.5 KiB	—	15.1 KiB	✅
Codex	Thread snapshot wire	—	5.7 KiB	—	7.3 KiB	✅
Codex	Live turn WebSocket wire	—	5.7 KiB	—	7.8 KiB	✅
Codex	Live turn WebSocket decoded	—	50.4 KiB	—	66.4 KiB	✅
Codex	Live turn messages	—	11	—	21	✅
Claude	Total thread wire	—	11.4 KiB	—	15.1 KiB	✅
Claude	Thread snapshot wire	—	5.7 KiB	—	7.3 KiB	✅
Claude	Live turn WebSocket wire	—	5.7 KiB	—	7.8 KiB	✅
Claude	Live turn WebSocket decoded	—	51.2 KiB	—	66.4 KiB	✅
Claude	Live turn messages	—	11	—	21	✅

Baseline: unavailable · PR result: e78dde7 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 95.6 KiB
• Claude decoded thread snapshot: 96.3 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.