Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions docs/contributing/disaster-recovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,13 +122,17 @@ Restore rebuilds these; do not treat them as recovery media:
sealed dump. Storage ids absent from the inventory are not deleted by restore.
- R2 restore puts sealed objects back by key; it does not sweep orphans that
appeared after the sealed day.
- **StorageRunner inventory still reads `package_runtime_runs` in D1** for
storage ids (plus jobs, archived artifacts, app packages, and service names).
New runs write storage ids only into `RunLog`, so a StorageRunner bucket
referenced solely by a post-migration run record is not yet picked up by the
platform DR inventory. Account deletion/export union `RunLog` storage ids for
purge/portability; extending DR inventory the same way is a separate follow-up
if those orphan-only buckets matter for sealed-day completeness.
- **StorageRunner inventory** unions authoritative D1 sources: `jobs`,
`archived_job_artifacts`, `saved_packages` (app packages), the
`user_storage_buckets` registry (including ad-hoc / execute buckets), and
`package_service_states` (projected service storage ids). Platform DR has only
a `D1Database`, so it does **not** walk package manifests or enumerate
`RunLog` Durable Objects. A service whose Durable Object never projected into
`package_service_states` is therefore absent from sealed-day inventory until
it heartbeats or transitions; account deletion/export cover those via manifest
enumeration. Buckets known only inside a user's `RunLog` (and never registered
in `user_storage_buckets` or an entity table) remain outside DR inventory by
design — RunLog is observability, not a canonical store.

## Credentials and Access

Expand Down
80 changes: 80 additions & 0 deletions packages/worker/migrations/0097-user-storage-buckets.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
-- Authoritative per-user durable storage bucket ownership.
--
-- "Which storage buckets does this user own?" is state, not history. Deriving
-- it from `package_runtime_runs` was wrong: that table stopped being written
-- (RunLog migration) and will drain under the 30-day retention policy. Ad-hoc
-- buckets (caller-supplied `storageId` on execute / storage capabilities) have
-- no other D1 record, so they would become unenumerable for backup, account
-- export, and account deletion.
--
-- This table is written on mutating StorageRunner access. The
-- `package_runtime_runs` arm below is a one-time rescue of pre-migration
-- buckets and is why this migration could not wait for the legacy drop.

CREATE TABLE IF NOT EXISTS user_storage_buckets (
user_id TEXT NOT NULL,
storage_id TEXT NOT NULL,
kind TEXT NOT NULL CHECK (kind IN ('job', 'app', 'service', 'execute', 'unknown')),
created_at TEXT NOT NULL,
last_seen_at TEXT NOT NULL,
PRIMARY KEY (user_id, storage_id)
);

CREATE INDEX IF NOT EXISTS idx_user_storage_buckets_user
ON user_storage_buckets(user_id);

INSERT OR IGNORE INTO user_storage_buckets (
user_id, storage_id, kind, created_at, last_seen_at
)
SELECT
user_id,
storage_id,
'job',
created_at,
updated_at
FROM jobs
WHERE storage_id IS NOT NULL AND trim(storage_id) != '';

INSERT OR IGNORE INTO user_storage_buckets (
user_id, storage_id, kind, created_at, last_seen_at
)
SELECT
user_id,
storage_id,
'job',
created_at,
updated_at
FROM archived_job_artifacts
WHERE storage_id IS NOT NULL AND trim(storage_id) != '';

INSERT OR IGNORE INTO user_storage_buckets (
user_id, storage_id, kind, created_at, last_seen_at
)
SELECT
user_id,
id,
'app',
created_at,
updated_at
FROM saved_packages
WHERE has_app = 1;

-- One-time rescue of pre-migration buckets (including ad-hoc execute/storage
-- ids) that only survive in run history until ~2026-08-25 retention drain.
INSERT OR IGNORE INTO user_storage_buckets (
user_id, storage_id, kind, created_at, last_seen_at
)
SELECT
user_id,
storage_id,
CASE surface
WHEN 'job' THEN 'job'
WHEN 'service' THEN 'service'
WHEN 'app_fetch' THEN 'app'
WHEN 'app_realtime' THEN 'app'
ELSE 'unknown'
END,
COALESCE(started_at, created_at, CURRENT_TIMESTAMP),
COALESCE(updated_at, finished_at, started_at, created_at, CURRENT_TIMESTAMP)
FROM package_runtime_runs
WHERE storage_id IS NOT NULL AND trim(storage_id) != '';
1 change: 1 addition & 0 deletions packages/worker/src/app/account-data-targets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ export const accountUserDataTargets: ReadonlyArray<UserScopedDataTarget> = [
{ kind: 'user_id', table: 'package_runtime_logs' },
{ kind: 'user_id', table: 'package_runtime_runs' },
{ kind: 'user_id', table: 'package_service_states' },
{ kind: 'user_id', table: 'user_storage_buckets' },
{ kind: 'user_id', table: 'usage_rollups' },
{ kind: 'user_id', table: 'user_activation_milestones' },
{ kind: 'user_id', table: 'user_package_run_successes' },
Expand Down
Loading
Loading