feat(billing): persist authoritative subscription observations

CHANGELOG.md

@@ -23,6 +23,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Security
 
 - Made contextual-orchestrator briefing requests fail closed unless an authenticated endpoint is configured. Deterministic generated text is restricted to explicit `SCOPEWEAVE_DEV=1`, message/provider responses are bounded and validated, and non-loopback HTTP transport is rejected.
+- Persist authoritative Stripe Subscription reads as normalized append-only
+  tenant-bound observations without mutating local entitlement state; atomic
+  savepoint writes now preserve the causal failure and never release an
+  unconfirmed rollback, preventing failed observation writes from being
+  accidentally committed during cleanup failure.
 - Added a bounded authoritative Stripe Subscription read boundary that validates
   exact subscription and tenant identity, normalizes immutable provider lifecycle
   facts without granting entitlement, and propagates `orgId` onto the underlying

docs/doctoring/stripe-subscription-observation-ledger.md

@@ -0,0 +1,95 @@
+# Authoritative Stripe subscription observation ledger
+
+## Status and authority
+
+**Status: active stacked PR evidence, not protected-`develop` shipped truth.**
+
+This record belongs to PR #526 and is stacked on PR #525's authoritative Stripe Subscription read boundary. Protected `develop` remains the shipped authority until the prerequisite stack is independently reviewed, protected-integrated, and revalidated against the final exact heads.
+
+Issue #488 remains open for lifecycle projection, monotonic entitlement policy, invoice/payment state, operator reconciliation, retention/export controls, and release acceptance. This slice deliberately persists provider-read evidence only; it does not grant or revoke an organization plan.
+
+## Buyer and data-integrity objective
+
+Stripe documents that webhook delivery order is not guaranteed and recommends retrieving provider objects when required to recover authoritative state. ScopeWeave therefore separates three concerns:
+
+1. cryptographically verified webhook delivery evidence;
+2. a tenant-verified current Subscription read from Stripe;
+3. an append-only local observation ledger that preserves each accepted provider read without treating arrival order as entitlement authority.
+
+The ledger gives an operator or later reconciliation policy a durable chronology of what ScopeWeave actually observed. It never updates `orgs.plan` and never converts Stripe status directly into local authorization.
+
+## Normalized data model
+
+`server/stripe_subscription_observation_ledger.mjs` installs five normalized relations at database bootstrap:
+
+- `billing_stripe_customers`: one Stripe customer identity bound permanently to one ScopeWeave organization;
+- `billing_stripe_subscriptions`: one Stripe subscription identity bound permanently to one customer;
+- `billing_stripe_prices`: deduplicated provider price identities;
+- `billing_stripe_subscription_observations`: append-only provider lifecycle facts for one subscription read;
+- `billing_stripe_subscription_observation_prices`: ordered many-to-many observation/price membership.
+
+The design avoids storing organization plan/entitlement state in an observation row, avoids repeating customer/tenant facts in every observation, and preserves source-event provenance separately through an optional foreign key to `billing_stripe_webhook_events`. All newly owned database objects use descriptive multiword `snake_case` names.
+
+## Persistence invariants
+
+`recordAuthoritativeObservation(...)` validates the provider snapshot again at the persistence boundary even though PR #525 already normalizes the remote response. A valid record requires:
+
+- a positive safe-integer organization ID that exists locally;
+- bounded provider customer/subscription identifiers;
+- one of the explicitly accepted Stripe Subscription statuses;
+- Boolean cancel-at-period-end state;
+- safe nonnegative provider timestamps with end not preceding start;
+- one to 100 bounded price identifiers;
+- optional bounded invoice and previously persisted source-event identifiers.
+
+A previously seen Stripe Customer cannot be rebound to a different ScopeWeave organization, and a previously seen Subscription cannot be rebound to a different Customer. Identity conflicts fail closed with stable conflict semantics before a new observation is accepted.
+
+Successful provider reads append observations rather than updating old snapshots. The local observation timestamp is monotonic per subscription even if the host wall clock moves backward; `observation_id` remains the durable append order when timestamps tie.
+
+## Transaction and failure semantics
+
+Customer identity, subscription identity, prices, the observation row, and ordered price memberships are written under one SQLite savepoint. A forced downstream junction-row failure is covered by a realistic trigger regression that proves all preceding mutations roll back together.
+
+A distinct cleanup-failure regression was added after working-path comparison with the calendar-subscription persistence adapter exposed the same transaction hazard: an unconditional `RELEASE` in a `finally` block can commit the outermost savepoint when `ROLLBACK TO` itself failed. SQLite documents that releasing the outermost savepoint is equivalent to commit. The observation repository now:
+
+- preserves the causal operation error;
+- releases the savepoint only after rollback is confirmed;
+- suppresses cleanup-release errors after a confirmed rollback so they cannot replace the business/persistence failure;
+- leaves an unconfirmed failed savepoint open rather than risk committing partial state.
+
+The RED contributor head `f337b90ef1803290efb7e7df02745c7280e9d5de` added the cleanup regression and caused hosted `unit-and-api` to fail. The narrow production repair at `dada7f0ef3327fc16e0b0d02f270196285fafcd1` restored hosted `unit-and-api`, API, dependency-review, and OSV success without weakening a gate.
+
+## Tenant, privacy, and entitlement boundary
+
+The ledger stores only provider identifiers and lifecycle facts required for reconciliation. It does not store Stripe secret keys, signed raw webhook bodies, session credentials, arbitrary provider error bodies, or local entitlement decisions.
+
+Organization authority is purpose-bound: the local organization must already exist, the authoritative provider reader must have verified `Subscription.metadata.orgId`, and the persistence layer permanently binds the resulting Stripe identities to that tenant. A later policy layer must independently decide which observed provider state authorizes a local plan transition.
+
+The optional `source_event_id` is audit/reconciliation-trigger provenance only. Event arrival time is not a lifecycle ordering key and source-event presence never grants entitlement.
+
+## Acceptance trace
+
+Executable evidence includes:
+
+- `tests/unit/stripe-subscription-observation-ledger.test.mjs` for normalized schema shape, tenant/customer identity non-rebinding, append-only observations, validation, rollback atomicity, savepoint-cleanup failure, monotonic timestamps, source-event existence, and no direct plan mutation;
+- `server/db.mjs` for bootstrap-only schema installation after the verified Stripe event ledger;
+- `tests/unit/coverage-script-contract.test.mjs` for canonical coverage registration;
+- `package.json` for normal unit execution and owned-production c8 instrumentation.
+
+PR #523 is still the repository-owned exact-contributor-head checkout-control prerequisite for merge-grade evidence across this billing stack. Repository-native successes on a stack that still inherits the prior synthetic pull-request merge checkout behavior are useful causal evidence but are not promoted to final exact-head proof.
+
+## Rollback and recovery
+
+Before protected integration, rollback removes the observation schema/bootstrap wiring, repository, focused tests, coverage registration, this doctoring record, and the corresponding active-PR changelog entry together.
+
+After this ledger is eventually protected-shipped, rollback must not destroy accumulated observation history merely to revert a later entitlement policy. Recovery should retain provider evidence, re-fetch authoritative Subscription state, append a new verified observation, and replay the explicitly versioned policy from a known local/provider point.
+
+## References
+
+SQLite. (n.d.). *Savepoints*. SQLite Documentation. https://sqlite.org/lang_savepoint.html
+
+Stripe. (n.d.). *Receive Stripe events in your webhook endpoint*. Stripe Documentation. https://docs.stripe.com/webhooks
+
+Stripe. (n.d.). *Retrieve a subscription*. Stripe API Reference. https://docs.stripe.com/api/subscriptions/retrieve
+
+Stripe. (n.d.). *Using webhooks with subscriptions*. Stripe Documentation. https://docs.stripe.com/billing/subscriptions/webhooks

package.json

@@ -13,9 +13,9 @@
     "coverage": "npm run test:coverage",
     "server": "node server/server.mjs",
     "test:api": "node tests/api/auth-secret.test.mjs && node --env-file=tests/api/smoke.env tests/api/smoke.mjs && node tests/api/ratelimit.test.mjs && node tests/api/attachment-status.test.mjs && node tests/api/session-revocation.test.mjs && node tests/api/orchestrator-attribution.test.mjs && node tests/api/billing-checkout.test.mjs && node tests/api/billing-live-checkout.test.mjs && node tests/api/stripe-webhook.test.mjs",
-    "test:unit": "node tests/unit/opencode-config.test.mjs && node tests/unit/changelog-release-notes.test.mjs && node tests/unit/analytics.test.mjs && node tests/unit/cpm.test.mjs && node tests/unit/baseline-compare.test.mjs && node tests/unit/workload.test.mjs && node tests/unit/cost-evm.test.mjs && node tests/unit/msproject.test.mjs && node tests/unit/auth-password.test.mjs && node tests/unit/editor-unsaved.test.mjs && node tests/unit/static-coverage-evidence.test.mjs && node tests/unit/dep-types.test.mjs && node tests/unit/weekly-report.test.mjs && node tests/unit/clearfolio.test.mjs && node tests/unit/clearfolio-adapter-mock-hmac.test.mjs && node tests/unit/orchestrator.test.mjs && node tests/unit/orchestrator-coverage.test.mjs && node tests/unit/orchestrator-attribution.test.mjs && node tests/unit/sprint-stats.test.mjs && node tests/unit/burndown.test.mjs && node tests/unit/pm-analysis.test.mjs && node tests/unit/cloud-sync-security.test.mjs && node tests/unit/attachment-status.test.mjs && node tests/unit/clearfolio-status-signal.test.mjs && node tests/unit/coverage-script-contract.test.mjs && node tests/unit/billing-configuration.test.mjs && node tests/unit/billing-checkout-attempt.test.mjs && node tests/unit/billing-checkout-attempt-authority.test.mjs && node tests/unit/billing-checkout-reconciliation.test.mjs && node tests/unit/billing-checkout-reconciliation-authority.test.mjs && node tests/unit/billing-checkout.test.mjs && node tests/unit/billing-provider-boundary.test.mjs && node tests/unit/billing-checkout-review-regressions.test.mjs && node tests/unit/stripe-webhook-boundary.test.mjs && node tests/unit/stripe-webhook-event-ledger.test.mjs && node tests/unit/stripe-webhook-recorder-integration.test.mjs && node tests/unit/stripe-subscription-provider.test.mjs && node tests/unit/stripe-subscription-metadata-propagation.test.mjs && node tests/unit/toast-accessibility.test.mjs",
-    "test:coverage": "c8 --all --include=app.js --include=cloud-sync.js --include=scripts/ci/static_coverage_evidence.mjs --include=server/attachment_status.mjs --include=server/app.mjs --include=server/application_routes.mjs --include=server/auth.mjs --include=server/billing.mjs --include=server/billing_checkout_attempt.mjs --include=server/billing_configuration.mjs --include=server/clearfolio.mjs --include=server/orchestrator.mjs --include=server/stripe_webhook.mjs --include=server/stripe_webhook_event_ledger.mjs --include=server/stripe_subscription_provider.mjs --reporter=json --reporter=json-summary npm run test:coverage:cases",
-    "test:coverage:cases": "node tests/unit/coverage-script-contract.test.mjs && node tests/unit/attachment-status.test.mjs && node tests/unit/clearfolio-status-signal.test.mjs && node tests/unit/clearfolio-adapter-mock-hmac.test.mjs && node tests/unit/orchestrator.test.mjs && node tests/unit/orchestrator-coverage.test.mjs && node tests/unit/orchestrator-attribution.test.mjs && node tests/unit/msproject.test.mjs && node tests/unit/auth-password.test.mjs && node tests/unit/editor-unsaved.test.mjs && node tests/unit/static-coverage-evidence.test.mjs && node tests/unit/billing-configuration.test.mjs && node tests/unit/billing-checkout-attempt.test.mjs && node tests/unit/billing-checkout-attempt-authority.test.mjs && node tests/unit/billing-checkout-reconciliation.test.mjs && node tests/unit/billing-checkout-reconciliation-authority.test.mjs && node tests/unit/billing-checkout.test.mjs && node tests/unit/billing-provider-boundary.test.mjs && node tests/unit/billing-checkout-review-regressions.test.mjs && node tests/unit/stripe-webhook-boundary.test.mjs && node tests/unit/stripe-webhook-event-ledger.test.mjs && node tests/unit/stripe-webhook-recorder-integration.test.mjs && node tests/unit/stripe-subscription-provider.test.mjs && node tests/unit/stripe-subscription-metadata-propagation.test.mjs && npm run test:api",
+    "test:unit": "node tests/unit/opencode-config.test.mjs && node tests/unit/changelog-release-notes.test.mjs && node tests/unit/analytics.test.mjs && node tests/unit/cpm.test.mjs && node tests/unit/baseline-compare.test.mjs && node tests/unit/workload.test.mjs && node tests/unit/cost-evm.test.mjs && node tests/unit/msproject.test.mjs && node tests/unit/auth-password.test.mjs && node tests/unit/editor-unsaved.test.mjs && node tests/unit/static-coverage-evidence.test.mjs && node tests/unit/dep-types.test.mjs && node tests/unit/weekly-report.test.mjs && node tests/unit/clearfolio.test.mjs && node tests/unit/clearfolio-adapter-mock-hmac.test.mjs && node tests/unit/orchestrator.test.mjs && node tests/unit/orchestrator-coverage.test.mjs && node tests/unit/orchestrator-attribution.test.mjs && node tests/unit/sprint-stats.test.mjs && node tests/unit/burndown.test.mjs && node tests/unit/pm-analysis.test.mjs && node tests/unit/cloud-sync-security.test.mjs && node tests/unit/attachment-status.test.mjs && node tests/unit/clearfolio-status-signal.test.mjs && node tests/unit/coverage-script-contract.test.mjs && node tests/unit/billing-configuration.test.mjs && node tests/unit/billing-checkout-attempt.test.mjs && node tests/unit/billing-checkout-attempt-authority.test.mjs && node tests/unit/billing-checkout-reconciliation.test.mjs && node tests/unit/billing-checkout-reconciliation-authority.test.mjs && node tests/unit/billing-checkout.test.mjs && node tests/unit/billing-provider-boundary.test.mjs && node tests/unit/billing-checkout-review-regressions.test.mjs && node tests/unit/stripe-webhook-boundary.test.mjs && node tests/unit/stripe-webhook-event-ledger.test.mjs && node tests/unit/stripe-webhook-recorder-integration.test.mjs && node tests/unit/stripe-subscription-provider.test.mjs && node tests/unit/stripe-subscription-metadata-propagation.test.mjs && node tests/unit/stripe-subscription-observation-ledger.test.mjs && node tests/unit/toast-accessibility.test.mjs",
+    "test:coverage": "c8 --all --include=app.js --include=cloud-sync.js --include=scripts/ci/static_coverage_evidence.mjs --include=server/attachment_status.mjs --include=server/app.mjs --include=server/application_routes.mjs --include=server/auth.mjs --include=server/billing.mjs --include=server/billing_checkout_attempt.mjs --include=server/billing_configuration.mjs --include=server/clearfolio.mjs --include=server/orchestrator.mjs --include=server/stripe_webhook.mjs --include=server/stripe_webhook_event_ledger.mjs --include=server/stripe_subscription_provider.mjs --include=server/stripe_subscription_observation_ledger.mjs --reporter=json --reporter=json-summary npm run test:coverage:cases",
+    "test:coverage:cases": "node tests/unit/coverage-script-contract.test.mjs && node tests/unit/attachment-status.test.mjs && node tests/unit/clearfolio-status-signal.test.mjs && node tests/unit/clearfolio-adapter-mock-hmac.test.mjs && node tests/unit/orchestrator.test.mjs && node tests/unit/orchestrator-coverage.test.mjs && node tests/unit/orchestrator-attribution.test.mjs && node tests/unit/msproject.test.mjs && node tests/unit/auth-password.test.mjs && node tests/unit/editor-unsaved.test.mjs && node tests/unit/static-coverage-evidence.test.mjs && node tests/unit/billing-configuration.test.mjs && node tests/unit/billing-checkout-attempt.test.mjs && node tests/unit/billing-checkout-attempt-authority.test.mjs && node tests/unit/billing-checkout-reconciliation.test.mjs && node tests/unit/billing-checkout-reconciliation-authority.test.mjs && node tests/unit/billing-checkout.test.mjs && node tests/unit/billing-provider-boundary.test.mjs && node tests/unit/billing-checkout-review-regressions.test.mjs && node tests/unit/stripe-webhook-boundary.test.mjs && node tests/unit/stripe-webhook-event-ledger.test.mjs && node tests/unit/stripe-webhook-recorder-integration.test.mjs && node tests/unit/stripe-subscription-provider.test.mjs && node tests/unit/stripe-subscription-metadata-propagation.test.mjs && node tests/unit/stripe-subscription-observation-ledger.test.mjs && npm run test:api",
     "test:e2e": "playwright test",
     "test:e2e:headed": "playwright test --headed",
     "test:e2e:cloud": "playwright install chromium && playwright test tests/e2e/cloud.spec.js tests/e2e/toast-accessibility.spec.js",

server/db.mjs

@@ -13,6 +13,10 @@ import {
   createSqliteStripeWebhookEventRepository,
   installStripeWebhookEventSchema,
 } from './stripe_webhook_event_ledger.mjs';
+import {
+  createSqliteStripeSubscriptionObservationRepository,
+  installStripeSubscriptionObservationSchema,
+} from './stripe_subscription_observation_ledger.mjs';
 
 const __dirname = dirname(fileURLToPath(import.meta.url));
 const dbPath = process.env.SCOPEWEAVE_DB || join(__dirname, '..', 'data.db');
@@ -192,6 +196,8 @@ export const billingCheckoutAttempts = createSqliteBillingCheckoutAttemptReposit
 installStripeWebhookEventSchema(db);
 export const stripeWebhookEvents = createSqliteStripeWebhookEventRepository(db);
 configureStripeWebhookEventRecorder((evidence) => stripeWebhookEvents.recordVerifiedEvent(evidence));
+installStripeSubscriptionObservationSchema(db);
+export const stripeSubscriptionObservations = createSqliteStripeSubscriptionObservationRepository(db);
 
 // node:sqlite returns lastInsertRowid as number|bigint; normalize to Number.
 export const rowid = (r) => Number(r.lastInsertRowid);