Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
846e38d
fix account lifecycle isolation and cleanup
cursoragent Jul 22, 2026
f5aa452
fix lifecycle regression coverage
cursoragent Jul 22, 2026
d67150e
harden report redaction and R2 failures
cursoragent Jul 22, 2026
7aec307
fix independent account lifecycle review findings
cursoragent Jul 22, 2026
a844413
fix lifecycle test harness expectations
cursoragent Jul 22, 2026
2b0790b
sign R2 export cursors
cursoragent Jul 22, 2026
5580079
quiesce writes and sweep historical assets
cursoragent Jul 22, 2026
79b92f1
align lifecycle guards with write boundaries
cursoragent Jul 22, 2026
01c856d
update write guard test fixtures
cursoragent Jul 22, 2026
463be56
close in-flight asset write races
cursoragent Jul 22, 2026
089b42a
coordinate in-flight writes during deletion
cursoragent Jul 22, 2026
d9e8f11
expire stale write leases and guard outbound writes
cursoragent Jul 22, 2026
e8cb5cb
lease package invocations and sweep icon cache
cursoragent Jul 22, 2026
f3414cf
update invocation lease test harness
cursoragent Jul 22, 2026
e5dc291
split account write lease migration
cursoragent Jul 22, 2026
97b6103
test icon cache deletion race
cursoragent Jul 22, 2026
9e686e3
fix icon cache race fixture
cursoragent Jul 22, 2026
dcdb60a
model icon cache lease race
cursoragent Jul 22, 2026
9543a14
assert icon cache lease race outcome
cursoragent Jul 22, 2026
04e2db3
map invocation lease races to conflict
cursoragent Jul 22, 2026
e62b633
address final AI review feedback
cursoragent Jul 23, 2026
481994d
update strict email cleanup tests
cursoragent Jul 23, 2026
9251b78
renew active leases and scope avatar writes
cursoragent Jul 23, 2026
f9a8f64
align R2 manifest section key
cursoragent Jul 23, 2026
21a173e
lease user mutation services and sweep retriever cache
cursoragent Jul 23, 2026
4067d73
update mutation lease test harnesses
cursoragent Jul 23, 2026
07c635b
handle missing email blob binding explicitly
cursoragent Jul 23, 2026
e701715
complete merge-gate lifecycle coverage
cursoragent Jul 23, 2026
e78e4cf
update merge-gate test harnesses
cursoragent Jul 23, 2026
cc96d07
enforce MCP agent session ownership
cursoragent Jul 23, 2026
99b682d
scope MCP lease test identity
cursoragent Jul 23, 2026
3ca9d94
scope MCP session ownership lookup
cursoragent Jul 23, 2026
a2685ee
replace expiring leases and gate legacy MCP backfill
cursoragent Jul 23, 2026
f927e13
fix token lease compatibility and repair docs
cursoragent Jul 23, 2026
14e7fef
count canonical service storage ids
cursoragent Jul 23, 2026
a9e9360
close cold MCP web lease and DO export gaps
cursoragent Jul 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions docs/contributing/account-write-lease-repair.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Account write lease repair

Account write leases do not expire. A crashed writer intentionally blocks
account deletion until an administrator verifies that the process is gone and
releases its exact token with an audit reason.

First inspect active leases:

```javascript
await kody.admin_account_write_lease_list({
stable_user_id: 'user-id-from-admin-account-lookup',
})
```

Review the returned token, holder, and `acquired_at` timestamp against runtime
logs before repair. Then release exactly that inspected lease:

```javascript
await kody.admin_account_write_lease_repair({
stable_user_id: 'user-id-from-admin-account-lookup',
token: '00000000-0000-4000-8000-000000000000',
expected_acquired_at: '2026-07-23 05:00:00',
reason: 'Confirmed worker invocation terminated; no process remains.',
})
```

The repair writes `account_write_lease_repairs` before releasing the token.
Wrong-user, stale timestamp, already-released token, or short reason requests
fail closed. Retry account deletion only after inspection shows no active
leases.
95 changes: 58 additions & 37 deletions docs/contributing/architecture/data-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,18 @@ installs and ordinary forks share the same row shape and therefore appear as
## Account deletion inventory

Account deletion is implemented in `packages/worker/src/app/account-deletion.ts`
and is intentionally inventory driven. The operation first enumerates user-owned
identifiers while D1 rows still exist, then best-effort deletes out-of-band
stores, then deletes or clears D1 rows, revokes OAuth grants, and finally
deletes the `users` row. Each step records deleted counts, updated counts for
cleared references, and warnings so the HTTP response states what was removed
and what needs operator attention. Re-running the operation is safe: missing
rows, missing KV keys, missing vectors, deleted Artifacts repos, and
already-cleared Durable Objects are treated as successful no-ops or warning-only
failures.
and is intentionally inventory driven. Before inventory it durably sets
`users.deleting_at`; browser, MCP, package-invocation, and job mutation
boundaries then reject writes, while the deletion route can still authenticate
the marked account for retry. The operation performs idempotent out-of-band and
OAuth cleanup. Any critical cleanup failure preserves D1, the marker, and the
user row for retry. Only after cleanup succeeds does one atomic D1 batch delete
or clear all user rows and the `users` row. Each step records deleted counts,
updated counts for cleared references, and warnings so the HTTP response states
what was removed and what needs operator attention. Re-running the operation is
safe: missing rows, missing KV keys, missing vectors, deleted Artifacts repos,
and already-cleared Durable Objects are treated as successful no-ops or
warning-only failures.

System email rows owned by `system:email` are intentionally excluded from
account deletion. They are operator-owned inbound mail for reserved platform
Expand All @@ -76,19 +79,19 @@ Deletion must cover these user-owned surfaces:
- **Durable Objects:** `JobManager`, `StorageRunner`, `RepoSession`,
`RemoteConnectorSession`, `PackageRealtimeSession`, and
`PackageServiceInstance` are purged through account-deletion RPCs after their
D1 identifiers are collected. `MCP` objects are session-keyed by the MCP SDK
rather than user-keyed and are not globally enumerable; account deletion
revokes OAuth grants/tokens so those sessions cannot continue making
authorized user requests.
D1 identifiers are collected. `MCP` objects remain SDK session-keyed, while
`mcp_agent_sessions` indexes each Durable Object id by authenticated stable
user id so account deletion can purge stored props, conversation state,
raw-fetch state, and transport storage before revoking OAuth grants.
- **Vectorize:** memory, job, and saved-package vector ids are derived from D1
rows and removed with `deleteByIds`.
- **R2:** raw email MIME blobs in `EMAIL_BLOBS` are enumerated from
`email_messages` (deterministic `emailRawMimeKey` keys) and attachment storage
keys while those rows still exist, then deleted best-effort with warnings,
mirroring the KV/Vectorize reporting. Rows owned by `system:email` keep their
blobs here (they are not user data); those blobs are removed when the
system-email retention prune deletes the messages through the shared
delete-message helper.
keys while those rows still exist. A failed object delete aborts D1
finalization so those inventory rows remain available for retry. Rows owned by
`system:email` keep their blobs here (they are not user data); those blobs are
removed when the system-email retention prune deletes the messages through the
shared delete-message helper.
- **KV:** published bundle artifact keys, source/manifest snapshot keys,
community listing snapshots, and per-user package retriever cache/index keys
in `BUNDLE_ARTIFACTS_KV` are deleted before D1 projection rows are removed.
Expand Down Expand Up @@ -148,8 +151,10 @@ hashes, and package invocation token hashes. The manifest states these
redactions explicitly so a partial or intentionally redacted export is not
mistaken for a complete secret backup.

The browser route `GET /account/export.json` downloads a full JSON export for
the signed-in user. The MCP capability domain `account` provides a
The browser route `GET /account/export.json` downloads a bounded metadata
manifest for the signed-in user and identifies the MCP capabilities required for
a complete export. It deliberately does not inline D1 rows, Durable Object
state, or R2 bytes. The MCP capability domain `account` provides the complete,
migration-safe chunked interface:

- `account_export_manifest` returns the manifest, counts, warnings, and chunking
Expand All @@ -158,13 +163,19 @@ migration-safe chunked interface:
with `section: "d1_table"` and a table name. Durable storage buckets are read
with `section: "storage_runner"` and a `storage_id`, using the same
StorageRunner `exportStorage({ pageSize, startAfter })` RPC as the dedicated
storage export capability.

D1 rows are always read with SQL-level keyset pagination: every query orders by
the table's `rowid`, resumes strictly after an opaque cursor, and applies a SQL
`LIMIT`, so a single query never loads a whole table. `account_export_section`
fetches only the requested page, and the full browser export streams each table
page by page (500 rows per query) before assembling the final JSON document.
storage export capability. R2 raw MIME, attachment, avatar, and icon objects
use `section: "r2_object"`; each response contains at most one 256 KiB base64
chunk and an opaque cursor. Each request uses bounded `LIMIT 1` ownership
queries rather than reconstructing inventory. Continuation cursors bind the
source row, object key, size, and ETag; ownership/key mutations and object
overwrites are reported instead of mixing generations. Missing objects are
represented explicitly.

D1 manifest counts use bounded SQL `COUNT(*)` queries. D1 section rows are read
with SQL-level keyset pagination: every query orders by the table's `rowid`,
resumes strictly after an opaque cursor, and applies a SQL `LIMIT`, so a single
query never loads a whole table. `account_export_section` fetches only the
requested page.

Durable Object export behavior:

Expand Down Expand Up @@ -352,8 +363,8 @@ rows only.
that canonical key): `deleteEmailMessageById` (best-effort after the D1 row
delete), user email retention and system-email retention (strict: blob delete
before row delete; failed blob deletes skip the row for retry), and account
deletion (best-effort with warnings; enumerates every message id for the user
before D1 rows are removed).
deletion (strict before atomic D1 finalization; a failed blob delete preserves
every message row for retry).
- Bucket names: `kody-email-blobs` (production), per-preview
`{worker}-email-blobs` buckets created and cleaned up by
`tools/ci/preview-resources.ts`, and the test env reuses the preview-style
Expand Down Expand Up @@ -683,7 +694,8 @@ to `durableObjectNameFromParts`).
- `RemoteConnectorSession`:
`idFromName(JSON.stringify([userId.trim(), normalizedInstanceId]))`.
- `MCP`: session-keyed by the MCP SDK rather than by user id; OAuth caller
context is the request-time ownership boundary.
context is the request-time ownership boundary and `mcp_agent_sessions`
provides deletion-only enumeration by stable user id.

Storage ids are also stable strings: execute storage uses `exec:{uuid}`, job
storage uses `job:{jobId}`, and package services use
Expand All @@ -702,10 +714,13 @@ app-owned keys in it. App-owned `BUNDLE_ARTIFACTS_KV` keys are:
- `package-retriever-index-entry:v1:{userId}:{scope}:{packageId}:{retrieverKey}`
for per-entry retriever index rows.
- `package-retriever-index:v1:{userId}:{scope}` — legacy combined retriever
index blob (no longer written). Residual blobs for active users are unused and
are not removed by refresh/removal; account deletion lists and deletes keys
under the user-scoped prefix `package-retriever-index:v1:{userId}:`. There is
no global sweep because that would require cross-user KV enumeration.
index blob (no longer written). Package refresh/removal and account deletion
delete the known `search` and `context` keys directly, so cleanup does not
depend on KV prefix listing. There is no global sweep because that would
require cross-user KV enumeration.
- `derived-cache:v1:usage-rollups:user:{userId}:asof:{YYYY-MM}` — derived
per-user usage read model written with KV `expirationTtl`; retention is five
minutes, so immediate account-deletion cleanup is not required.

Account deletion derives these keys from D1 rows and package ids before deleting
D1 projections. New KV prefixes must add corresponding account-deletion coverage
Expand All @@ -717,16 +732,22 @@ App-owned R2 keys are:

- `community-icon:v1/{listingId}/{commit}/asset` — processed public community
icon bytes at the listing's pinned or icon commit. The listing id is the
public ownership boundary; account deletion derives keys from the owner's
listing rows and their joined package source commits.
public ownership boundary. Account deletion paginates and strictly deletes
every key under each D1-owned listing prefix, including historical revisions.

- `user-avatars/{stableUserId}/{contentHash}.{extension}` — profile avatars.
Account deletion paginates and strictly deletes the complete stable-user
prefix, including historical replacements left by earlier cleanup failures.

- `email-raw:v1:{userId}/{messageId}` — raw email MIME for the message row that
stores this key in `email_messages.raw_mime_key`. The `userId` prefix is part
of the per-user isolation contract; account deletion deletes a user's blobs by
the keys stored on their rows.

New R2 key prefixes must add corresponding account-deletion coverage or a
deliberate retention note, same as KV.
deliberate retention note, same as KV. All currently registered R2 surfaces use
the bounded `r2_object` account-export section; the inventory is derived from
the same user-owned D1 rows used by account deletion.

### Vectorize metadata contracts

Expand Down
44 changes: 44 additions & 0 deletions docs/contributing/mcp-agent-session-backfill.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# MCP Agent session backfill

Migration `0086` indexes MCP Agent Durable Object ids by stable user id. Before
account deletion may run in an environment, dormant objects created before that
migration must be indexed and the version-1 completion marker must be written.

The operator sweep uses Cloudflare's Durable Objects list API and the existing
`CAPABILITY_REINDEX_SECRET` maintenance authentication. Never pass secrets as
CLI flags or commit audit output.

## Required environment

- `CLOUDFLARE_ACCOUNT_ID`
- `CLOUDFLARE_API_TOKEN` with Workers/Durable Objects read access
- `CAPABILITY_REINDEX_SECRET`

## Dry run

```bash
npm run backfill:mcp-agent-sessions -- \
--origin https://heykody.dev \
--worker-script kody \
--audit-out /tmp/mcp-agent-session-backfill-dry-run.json
```

The command resolves the `MCP` namespace, cursor-pages every object with stored
data, and calls the maintenance ownership RPC in batches of 50 without writing
index rows. Review every `no_owner`, conflict, and failure before execution.

## Execute

```bash
npm run backfill:mcp-agent-sessions -- \
--origin https://heykody.dev \
--worker-script kody \
--audit-out /tmp/mcp-agent-session-backfill-execute.json \
--execute
```

Execution is idempotent. It registers same-owner rows, rejects ownership
conflicts, writes an audit artifact, and writes completion marker version `1`
only when no failures remain. Account deletion fails closed with HTTP 503 until
that marker exists. Do not run the production execute command from an agent; an
operator must review the dry-run artifact first.
15 changes: 11 additions & 4 deletions docs/use/privacy.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,12 @@ related social surfaces. When visibility is **private**, the public profile is
not found, others cannot follow you, and you are omitted from stargazer lists
and other users' timelines. See [Community profiles](./community-profiles.md).

Account export includes your profile columns and social rows where you are on
either side of an edge (follows as follower or followee; stars you placed or
that sit on listings you own; activity you authored or that is tied to your
listings). Account deletion removes those same rows.
Account export includes your profile columns and social rows where you are a
participant (follows as follower or followee, stars you placed, and activity you
authored). The browser download is a bounded metadata manifest; use its
`account_export_section` instructions to retrieve every D1, Durable Object, and
R2 page for a complete portable export. Account deletion removes those same
user-owned rows and objects.

## What a deployment admin can see

Expand Down Expand Up @@ -81,6 +83,11 @@ reviewer identity, notes, and timestamps are not included. If an admin who
reviewed your feedback deletes their account, Kody clears that reviewer's
attribution while retaining your submission for the lifecycle described above.

Community stars, stored activity events, ratings, forks, and reports appear only
in the participating user's account export. Owning the related listing does not
expose another user's event timestamps or types, identity, rating or adoption
notes, report reasons, or moderation details.

## What an admin can never see

The admin role is not a general data-access role. Approving platform feedback
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
"primitives:check": "node .agents/skills/visual-recap/scripts/check-primitives-map.mjs",
"primitives:classify": "node .agents/skills/visual-recap/scripts/classify-primitives.mjs",
"migrations:check": "node tools/check-migrations.ts",
"backfill:mcp-agent-sessions": "node tools/ci/backfill-mcp-agent-sessions-cli.ts",
"migrate:local": "node --env-file=packages/worker/.env ./wrangler-env.ts d1 migrations apply APP_DB --local",
"migrate:e2e": "node --env-file=packages/worker/.env ./wrangler-env.ts d1 migrations apply APP_DB --local --persist-to .wrangler/state/e2e",
"preview": "nx run worker:build-client && npm run migrate:local && node --env-file=packages/worker/.env ./wrangler-env.ts dev --local",
Expand Down
5 changes: 5 additions & 0 deletions packages/worker/migrations/0084-account-deleting-state.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
ALTER TABLE users ADD COLUMN deleting_at TEXT;

CREATE INDEX idx_users_deleting_at
ON users(deleting_at)
WHERE deleting_at IS NOT NULL;
2 changes: 2 additions & 0 deletions packages/worker/migrations/0085-account-write-leases.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
ALTER TABLE users ADD COLUMN active_write_count INTEGER NOT NULL DEFAULT 0;
ALTER TABLE users ADD COLUMN active_write_expires_at TEXT;
8 changes: 8 additions & 0 deletions packages/worker/migrations/0086-mcp-agent-sessions.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
CREATE TABLE mcp_agent_sessions (
do_id TEXT PRIMARY KEY NOT NULL,
user_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (CURRENT_TIMESTAMP)
);

CREATE INDEX idx_mcp_agent_sessions_user_id
ON mcp_agent_sessions(user_id);
22 changes: 22 additions & 0 deletions packages/worker/migrations/0087-account-write-lease-tokens.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
CREATE TABLE account_write_leases (
token TEXT PRIMARY KEY NOT NULL,
user_id TEXT NOT NULL,
holder TEXT NOT NULL,
acquired_at TEXT NOT NULL,
released_at TEXT
);

CREATE INDEX idx_account_write_leases_active_user
ON account_write_leases(user_id)
WHERE released_at IS NULL;

CREATE TABLE account_write_lease_repairs (
id TEXT PRIMARY KEY NOT NULL,
target_user_id TEXT NOT NULL,
lease_token TEXT NOT NULL,
lease_holder TEXT NOT NULL,
lease_acquired_at TEXT NOT NULL,
repaired_by_user_id TEXT NOT NULL,
reason TEXT NOT NULL,
created_at TEXT NOT NULL
);
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
CREATE TABLE deployment_backfill_markers (
key TEXT PRIMARY KEY NOT NULL,
version INTEGER NOT NULL,
completed_at TEXT NOT NULL,
audit_summary_json TEXT NOT NULL
);
18 changes: 18 additions & 0 deletions packages/worker/src/app/account-data-targets.node.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -159,3 +159,21 @@ test('account export redaction maps stay centralized in the D1 registry', () =>
])
expect(accountExportRedactedForeignUserId).toBe('[redacted]')
})

test('listing-owner deletion predicates are excluded from export for private participation rows', () => {
const excludedListingChildren = accountUserDataTargets
.filter(
(target) =>
target.kind === 'community_listing_child' &&
target.includeInExport === false,
)
.map((target) => target.table)
.sort()
expect(excludedListingChildren).toEqual([
'community_activity_events',
'community_forks',
'community_ratings',
'community_reports',
'community_stars',
])
})
Loading
Loading