feat(services): shadow liveness in UserMeter

docs/contributing/architecture/data-storage.md

@@ -172,16 +172,16 @@ migration-safe chunked interface:
   with `section: "storage_runner"` and a `storage_id`, using the same
   StorageRunner `exportStorage({ pageSize, startAfter })` RPC as the dedicated
   storage export capability. User meter counters use `section: "user_meter"` and
-  the `UserMeter.exportCounters` RPC (daily counters plus additive
-  `storageBytesShadow` on the first page only when present; explicitly
-  non-authoritative). Mailbox metadata uses `section: "mailbox"` and the
-  `Mailbox.exportMailbox` RPC. R2 raw MIME, attachment, avatar, and icon objects
-  use `section: "r2_object"`; each response contains at most one 256 KiB base64
-  chunk and an opaque cursor. Each request uses bounded `LIMIT 1` ownership
-  queries rather than reconstructing inventory. Continuation cursors bind the
-  source row, object key, size, and ETag; ownership/key mutations and object
-  overwrites are reported instead of mixing generations. Missing objects are
-  represented explicitly.
+  the `UserMeter.exportCounters` RPC (daily counters plus additive shadow fields
+  on the first page only when present: `storageBytesShadow` and
+  `packageServiceStatesShadow`; explicitly non-authoritative). Mailbox metadata
+  uses `section: "mailbox"` and the `Mailbox.exportMailbox` RPC. R2 raw MIME,
+  attachment, avatar, and icon objects use `section: "r2_object"`; each response
+  contains at most one 256 KiB base64 chunk and an opaque cursor. Each request
+  uses bounded `LIMIT 1` ownership queries rather than reconstructing inventory.
+  Continuation cursors bind the source row, object key, size, and ETag;
+  ownership/key mutations and object overwrites are reported instead of mixing
+  generations. Missing objects are represented explicitly.
 
 D1 manifest counts use bounded SQL `COUNT(*)` queries. D1 section rows are read
 with SQL-level keyset pagination: every query orders by the table's `rowid`,
@@ -205,13 +205,15 @@ Durable Object export behavior:
   never pruned by retention. See [Run records](./run-records.md).
 - `UserMeter` exports daily entitlement counter rows through the `user_meter`
   section (`exportCounters` RPC; keyset pagination by UTC `day` and `resource`).
-  The same RPC may return additive `storageBytesShadow` on the first page only
-  (`startAfter` absent) when the schema-v4 shadow row exists (`null` on later
-  pages and when never shadowed); section totals count it as one row when
-  present, but it is explicitly **non-authoritative** — usage and enforcement
-  read D1 `users.d1_storage_bytes`. Retention is self-enforced inside the DO
-  (seven UTC days of counter and inbound-delivery-claim rows); shadow
-  storage-byte state is not time-pruned. See
+  The same RPC may return additive shadow fields on the first page only
+  (`startAfter` absent): `storageBytesShadow` when the schema-v4 row exists, and
+  `packageServiceStatesShadow` when schema-v5 service rows exist (`null` on
+  later pages and when never shadowed). Section totals count each shadow
+  inventory once when present, but both are explicitly **non-authoritative** —
+  usage and enforcement read D1 (`users.d1_storage_bytes` and
+  `package_service_states` respectively). Retention is self-enforced inside the
+  DO (seven UTC days of counter and inbound-delivery-claim rows); shadow
+  storage-byte and package-service liveness rows are not time-pruned. See
   [Entitlements](./entitlements.md#usermeter-expand-phase).
 - `Mailbox` exports per-user email metadata (threads, messages, attachments,
   delivery events) through the account-export `mailbox` section (`exportMailbox`
@@ -262,8 +264,9 @@ The schema is defined by migrations in `packages/worker/migrations/`:
   [Platform accounts](./platform-accounts.md)). `d1_storage_bytes` and
   `d1_storage_bytes_updated_at` (migration 0122) are the **sole authority** for
   D1 payload storage-byte read, enforcement, and reconciliation. UserMeter
-  `storage_bytes_state` (schema v4) is an optional expand-phase shadow only —
-  see [Entitlements](./entitlements.md#usermeter-expand-phase). Inbound email
+  `storage_bytes_state` (schema v4) and `package_service_states` (schema v5) are
+  optional expand-phase shadows only — see
+  [Entitlements](./entitlements.md#usermeter-expand-phase). Inbound email
   routing does not reverse-resolve stable ids at all — it uses the indexed
   username lookup (`findPublicUserIdentityByUsername`). Contextless paths
   resolve stable ids with one indexed point read on `users.stable_user_id` (for
@@ -293,8 +296,12 @@ The schema is defined by migrations in `packages/worker/migrations/`:
   [Run records](./run-records.md)). Execution history rows live in the same DO.
 - `package_service_states` (`0095-package-service-states.sql`): authoritative
   per-service liveness projection (`running` / `idle` / `stopped` / `error`) for
-  entitlement concurrency. Upserted and heartbeaten by the package-service
-  Durable Object; not derived from run history.
+  entitlement concurrency, discovery, and export/deletion inventory. Upserted
+  and heartbeaten (1h) by the `PackageServiceInstance` Durable Object; running
+  counts treat rows stale after 24h without a fresh heartbeat. Not derived from
+  run history. Expand-phase slice 4 Phase A also best-effort shadows each row
+  into the per-user `UserMeter` DO (schema v5); D1 remains sole authority in
+  that slice — see [Entitlements](./entitlements.md#usermeter-expand-phase).
 - `entity_sources`: durable mapping from user-facing entities to Artifacts repos
   and their latest published commit
 - `saved_packages`: package metadata/search projection derived from published
@@ -519,18 +526,20 @@ Daily rate-style entitlement counters and inbound email delivery-id idempotency
 live in a per-user `UserMeter` Durable Object with SQLite
 (`packages/worker/src/entitlements/user-meter-do.ts`). Schema v4 adds an
 optional `storage_bytes_state` singleton as a **best-effort shadow** of D1
-`users.d1_storage_bytes` for future cutover — D1 remains sole authority for
-reads, reserves, and reconciliation in the current additive slice. The Worker
-binding is `USER_METER` (class `UserMeter`; Wrangler SQLite migration tag `v21`
-via `new_sqlite_classes` in `packages/worker/wrangler.jsonc`).
+`users.d1_storage_bytes`. Schema v5 adds an optional `package_service_states`
+table as a **best-effort shadow** of D1 `package_service_states` for future
+cutover — D1 remains sole authority for reads, running counts, discovery, and
+`service_start` enforcement in expand-phase slice 4 Phase A. The Worker binding
+is `USER_METER` (class `UserMeter`; Wrangler SQLite migration tag `v21` via
+`new_sqlite_classes` in `packages/worker/wrangler.jsonc`).
 
 Naming matches `RunLog` and `JobManager`: one object per untrimmed stable MCP
 `userId` via `userMeterDurableObjectName(userId)` → `idFromName(userId)` in
 `packages/worker/src/user-scoped-durable-object-name.ts`. There is no `user_id`
 column inside the DO because the object identity is the user.
 
 SQLite ownership (schema version tracked in `user_meter_meta`; current version
-**4**):
+**5**):
 
 - `daily_counters` — authoritative UTC-day counters for `email_sends_per_day`,
   `email_receives_per_day`, `execute_calls_per_day`, and
@@ -547,12 +556,23 @@ SQLite ownership (schema version tracked in `user_meter_meta`; current version
   and optional reconcile shadows; never read for enforcement or usage display.
   StorageRunner bucket estimates stay outside this row (see
   [Entitlements](./entitlements.md#usermeter-expand-phase)).
+- `package_service_states` — per-service **shadow** of D1 liveness rows
+  (`package_id`, `service_name`, `status`, `started_at`, `source_updated_at`,
+  monotonic `revision`, `updated_at`; primary key `(package_id, service_name)`).
+  Added in schema v5. Populated by best-effort dual-writes from
+  `PackageServiceInstance` on every D1 projection/delete; monotonic on
+  `source_updated_at`. Never read for enforcement, running counts, discovery, or
+  usage display in expand-phase slice 4 Phase A. Cutover-support RPCs
+  (`listPackageServiceStates`, `countRunningPackageServices`,
+  `bootstrapPackageServiceStates`) mirror D1 semantics for future parity review
+  only.
 
 Retention is self-enforced inside the DO: every read/write path
 opportunistically deletes counter and claim rows older than seven UTC days
 (`userMeterDailyCounterRetentionDays`). Enforcement only needs the current day;
 the window covers timezone edge cases, recent account exports, and inbound
-retries. Shadow storage-byte state is not time-pruned.
+retries. Shadow storage-byte and package-service liveness rows are not
+time-pruned.
 
 **Expand-phase D1 mirrors (daily counters only):** enforcement and point reads
 are authoritative in UserMeter for daily counters. D1
@@ -571,11 +591,12 @@ writes cannot overwrite newer state. See
 daily paths never read D1 for enforcement.
 
 Account deletion calls `UserMeter.purge()` (one RPC per user, no D1 id scan;
-`deleteAll` clears counters, claims, and any shadow storage state). Account
-export pages `UserMeter.exportCounters` through the `user_meter` manifest
-section / `account_export_section` (daily counters plus additive
-`storageBytesShadow` on the first page only when present; shadow field is
-non-authoritative).
+`deleteAll` clears counters, claims, and all shadow state including storage
+bytes and package-service liveness). Account export pages
+`UserMeter.exportCounters` through the `user_meter` manifest section /
+`account_export_section` (daily counters plus additive `storageBytesShadow` and
+`packageServiceStatesShadow` on the first page only when present; shadow fields
+are non-authoritative).
 
 ## Durable Objects (`Mailbox`)
 
@@ -718,7 +739,9 @@ storage homes as follows:
   `package:{encodeURIComponent(packageId)}` via `buildPackageStorageId` /
   `packageStorage()`. Shared durable data for every package surface.
 - **Package coordination** — `PackageServiceInstance` DO holds lifecycle and
-  alarms only; durable data stays in package storage. App facets and
+  alarms only; durable data stays in package storage. Each lifecycle projection
+  dual-writes D1 `package_service_states` (authority) and a best-effort
+  UserMeter shadow (expand-phase slice 4 Phase A). App facets and
   package-internal DO namespaces are extra StorageRunner buckets under the
   package id, not a general actor model.
 - **Package jobs** — schedule metadata in D1 `jobs`; run-local scratch in
@@ -741,8 +764,8 @@ via `durableObjectNameFromParts`); domain helpers such as
   package activation counters/milestones). See [Run records](./run-records.md).
 - `UserMeter` — `userMeterDurableObjectName(userId)` → `idFromName(userId)`. One
   daily-entitlement meter DO per user (untrimmed stable id, same as `RunLog`),
-  plus optional schema-v4 D1 storage-byte shadow. See
-  [Entitlements](./entitlements.md#usermeter-expand-phase).
+  plus optional schema-v4 D1 storage-byte shadow and schema-v5 package-service
+  liveness shadow. See [Entitlements](./entitlements.md#usermeter-expand-phase).
 - `StripePlanRefresh` — `stripePlanRefreshDurableObjectName(userId)` →
   `idFromName(userId)`. One ephemeral, one-shot reconciliation alarm per user;
   checkout and subscription webhook activity arm it as a backstop to the
@@ -1079,7 +1102,8 @@ to `durableObjectNameFromParts`).
 - `JobManager`: `idFromName(userId)` (no trim).
 - `RunLog`: `idFromName(userId)` (no trim); one execution-history DO per user.
 - `UserMeter`: `idFromName(userId)` (no trim); one daily-entitlement meter DO
-  per user, plus optional schema-v4 D1 storage-byte shadow.
+  per user, plus optional schema-v4 D1 storage-byte shadow and schema-v5
+  package-service liveness shadow.
 - `StripePlanRefresh`: `idFromName(userId)` (no trim); one ephemeral billing
   reconciliation alarm DO per user.
 - `Mailbox`: `idFromName(userId)` (no trim); one email-metadata DO per user.

docs/contributing/architecture/entitlements.md

@@ -124,6 +124,12 @@ storage layout and naming are documented in [Data storage](./data-storage.md).
 `storage_bytes_state` singleton as a **best-effort shadow** for future cutover —
 it does not drive reads, reserves, or reconciliation in this additive slice.
 
+**D1 package service liveness** (`package_service_states`) stays authoritative
+for running counts, discovery, and `service_start` enforcement. UserMeter schema
+v5 adds an optional per-service shadow table as **best-effort future-cutover
+support only** — see
+[Package service liveness — UserMeter shadow](#package-service-liveness--usermeter-shadow-expand-phase-slice-4-phase-a).
+
 StorageRunner bucket `estimatedBytes` and the per-bucket inventory in
 `user_storage_buckets` stay a **separate** quota component. StorageRunner write
 chokepoints pass `getCurrent` as a check-only composed total (D1 payload bytes
@@ -215,11 +221,78 @@ successful absolute reconciliation also advances the expand-phase shadow.
 `readUserD1StorageBytes` only.
 
 **Account export and purge:** `UserMeter.exportCounters` may return additive
-non-authoritative `storageBytesShadow` on the first page only (`startAfter`
-absent) when the shadow row exists; subsequent pages return `null` so paged
-consumers never double-count it (still counted once in the `user_meter` section
-total). `UserMeter.purge()` clears counters, inbound delivery claims, and any
-shadow storage state via `deleteAll`.
+non-authoritative shadow fields on the first page only (`startAfter` absent):
+`storageBytesShadow` when the schema-v4 row exists, and
+`packageServiceStatesShadow` when schema-v5 service rows exist. Subsequent pages
+return `null` for each shadow so paged consumers never double-count them
+(section totals still count each shadow inventory once when present).
+`UserMeter.purge()` clears counters, inbound delivery claims, and all shadow
+state (storage bytes and package-service liveness) via `deleteAll`.
+
+### Package service liveness — UserMeter shadow (expand phase slice 4, Phase A)
+
+D1 `package_service_states` remains the **sole authority** for running-service
+**count**, **discovery**, and **`service_start` enforcement** in this PR.
+Nothing in Phase A switches those reads or the `assertWithinEntitlement` path
+for `package_services` / `persistent_package_services`.
+
+UserMeter schema **v5** adds a per-service `package_service_states` table inside
+the DO as **best-effort shadow / future-cutover support only** (`status`,
+`started_at`, monotonic `source_updated_at` from the D1 projection timestamp,
+`revision`, `updated_at`). User scope is the DO identity — there is no `user_id`
+column. Shadow rows are never read for usage display, entitlement enforcement,
+or account-deletion inventory in this slice.
+
+**Dual-write from `PackageServiceInstance`:** every D1 projection also attempts
+a best-effort UserMeter shadow on the same lifecycle surface:
+
+- lifecycle transitions and warm-start restore after upgrades
+  (`projectServiceStateToD1`)
+- running-service heartbeat alarms (1h `packageServiceStateHeartbeatMs`,
+  unchanged)
+- stop, error, and idle projections that clear `running`
+- purge (`deleteProjectedServiceState` deletes D1 then shadow before
+  `deleteAll`)
+
+D1 upsert/delete runs first; shadow RPCs are optional when `USER_METER` is
+unbound and failures log `package-service-user-meter-shadow-failed` without
+affecting the service path. Shadow upserts reject stale/out-of-order writes when
+`sourceUpdatedAt` is older than the existing shadow row so cold bootstrap cannot
+clobber fresher state.
+
+**Timing unchanged:** live services heartbeat D1 `updated_at` every **1 hour**
+(`packageServiceStateHeartbeatMs`). Running counts still treat rows as stale
+after **24 hours** without a fresh heartbeat (`packageServiceStateStaleMs` in
+`entitlements/service.ts`). The UserMeter cutover-support RPC
+`countRunningPackageServices` uses the same 24h window on shadow
+`source_updated_at` but is **not** wired to enforcement in Phase A.
+
+**Account export:** `UserMeter.exportCounters` returns additive
+`packageServiceStatesShadow` on the first page only (`startAfter` absent); later
+pages return `null`. Section totals count the shadow inventory once when
+present; the field is explicitly non-authoritative — authoritative liveness
+remains on D1.
+
+**Account purge:** `UserMeter.purge()` clears package-service shadow rows with
+the rest of DO state via `deleteAll`.
+
+### Future package-service authority flip (contract follow-up)
+
+A separate **high-risk contract PR** — not a merge blocker for Phase A — will
+flip running-service count/discovery/enforcement into UserMeter only after:
+
+1. at least one full **24h stale-window soak** with shadow/D1 parity review, and
+2. a **cold-bootstrap design** for accounts whose DO shadow is empty while D1
+   still holds rows (`bootstrapPackageServiceStates` / equivalent).
+
+Until that flip, shadow divergence is acceptable; D1 remains the contract. **D1
+likely stays the enumeration index** for account export, deletion, and admin
+discovery until an alternate inventory exists — UserMeter would become the
+running-count authority first, not a wholesale replacement for every D1 reader.
+
+**Remaining expand roadmap:** slice 4 Phase A (this shadow slice) is additive
+only; slice 5 — account-deletion write fencing — follows independently. Storage
+authority flip remains the separate contract follow-up above.
 
 ### Future storage authority flip (contract follow-up)
 
@@ -231,9 +304,10 @@ Only then do reads, reserves, and reconciliation switch to UserMeter-first with
 D1 as mirror/cursor. Until that flip, shadow divergence is acceptable; D1
 remains the contract.
 
-**Remaining UserMeter expand roadmap:** slice 4 — `package_service_states`
-running counts (service liveness); slice 5 — account-deletion write fencing.
-Storage authority flip is tracked separately as the contract follow-up above.
+**Remaining UserMeter expand roadmap:** slice 4 Phase A —
+`package_service_states` UserMeter shadow (this PR; D1 authority unchanged);
+slice 5 — account-deletion write fencing. Package-service and storage authority
+flips are separate high-risk contract follow-ups after soak/parity review.
 
 **Daily-counter mirror retirement:** dropping D1 `entitlement_daily_counters`
 waits until reporting-off-D1 work merges and mirror parity is verified in
@@ -379,12 +453,14 @@ Rules:
   running package services) are counted **directly from their source D1 tables
   at the enforcement point** via built-in counters in `service.ts`. They do not
   depend on any metering or rollup tables. Running package services are counted
-  from `package_service_states` (status `running` and freshly heartbeaten), not
-  from run-history rows — see [Run records](./run-records.md)
-  (`state-vs-history`). **Concurrent workflows** are authoritative in per-user
-  RunLog `workflow_projections`: create reserves atomically via
-  `reserveWorkflowProjectionSlot`, and usage readers call
-  `countActiveWorkflowProjections` through
+  from D1 `package_service_states` (status `running` and freshly heartbeaten; 1h
+  heartbeat, 24h staleness), not from run-history rows — see
+  [Run records](./run-records.md) (`state-vs-history`). Expand-phase slice 4
+  Phase A dual-writes the same projection into UserMeter as a non-authoritative
+  shadow; enforcement and `service_start` still read D1 only. **Concurrent
+  workflows** are authoritative in per-user RunLog `workflow_projections`:
+  create reserves atomically via `reserveWorkflowProjectionSlot`, and usage
+  readers call `countActiveWorkflowProjections` through
   `readCurrentEntitlementResourceUsage`. Expand-phase D1 `workflow_runs` is a
   compatibility mirror only.
 - **Rate-style limits** (email sends/receives per day, execute calls per day,

docs/contributing/architecture/primitives.yaml

@@ -516,7 +516,9 @@ primitives:
     name: User meter
     summary:
       Per-user UserMeter DO SQLite for daily entitlement counters, inbound
-      delivery idempotency, and expand-phase D1 storage-byte shadow.
+      delivery idempotency, expand-phase D1 storage-byte shadow, and
+      expand-phase package-service liveness shadow (D1 authority unchanged in
+      slice 4 Phase A).
     code:
       - packages/worker/src/entitlements/user-meter-do.ts
       - packages/worker/src/entitlements/user-meter-client.ts