diff --git a/CHANGELOG.md b/CHANGELOG.md index fc544cd4..3700da60 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Security - Made contextual-orchestrator briefing requests fail closed unless an authenticated endpoint is configured. Deterministic generated text is restricted to explicit `SCOPEWEAVE_DEV=1`, message/provider responses are bounded and validated, and non-loopback HTTP transport is rejected. +- Persist authoritative Stripe Subscription reads as normalized append-only + tenant-bound observations without mutating local entitlement state; atomic + savepoint writes now preserve the causal failure and never release an + unconfirmed rollback, preventing failed observation writes from being + accidentally committed during cleanup failure. - Added a bounded authoritative Stripe Subscription read boundary that validates exact subscription and tenant identity, normalizes immutable provider lifecycle facts without granting entitlement, and propagates `orgId` onto the underlying diff --git a/docs/doctoring/stripe-subscription-observation-ledger.md b/docs/doctoring/stripe-subscription-observation-ledger.md new file mode 100644 index 00000000..c16892c0 --- /dev/null +++ b/docs/doctoring/stripe-subscription-observation-ledger.md @@ -0,0 +1,95 @@ +# Authoritative Stripe subscription observation ledger + +## Status and authority + +**Status: active stacked PR evidence, not protected-`develop` shipped truth.** + +This record belongs to PR #526 and is stacked on PR #525's authoritative Stripe Subscription read boundary. Protected `develop` remains the shipped authority until the prerequisite stack is independently reviewed, protected-integrated, and revalidated against the final exact heads. + +Issue #488 remains open for lifecycle projection, monotonic entitlement policy, invoice/payment state, operator reconciliation, retention/export controls, and release acceptance. This slice deliberately persists provider-read evidence only; it does not grant or revoke an organization plan. + +## Buyer and data-integrity objective + +Stripe documents that webhook delivery order is not guaranteed and recommends retrieving provider objects when required to recover authoritative state. ScopeWeave therefore separates three concerns: + +1. cryptographically verified webhook delivery evidence; +2. a tenant-verified current Subscription read from Stripe; +3. an append-only local observation ledger that preserves each accepted provider read without treating arrival order as entitlement authority. + +The ledger gives an operator or later reconciliation policy a durable chronology of what ScopeWeave actually observed. It never updates `orgs.plan` and never converts Stripe status directly into local authorization. + +## Normalized data model + +`server/stripe_subscription_observation_ledger.mjs` installs five normalized relations at database bootstrap: + +- `billing_stripe_customers`: one Stripe customer identity bound permanently to one ScopeWeave organization; +- `billing_stripe_subscriptions`: one Stripe subscription identity bound permanently to one customer; +- `billing_stripe_prices`: deduplicated provider price identities; +- `billing_stripe_subscription_observations`: append-only provider lifecycle facts for one subscription read; +- `billing_stripe_subscription_observation_prices`: ordered many-to-many observation/price membership. + +The design avoids storing organization plan/entitlement state in an observation row, avoids repeating customer/tenant facts in every observation, and preserves source-event provenance separately through an optional foreign key to `billing_stripe_webhook_events`. All newly owned database objects use descriptive multiword `snake_case` names. + +## Persistence invariants + +`recordAuthoritativeObservation(...)` validates the provider snapshot again at the persistence boundary even though PR #525 already normalizes the remote response. A valid record requires: + +- a positive safe-integer organization ID that exists locally; +- bounded provider customer/subscription identifiers; +- one of the explicitly accepted Stripe Subscription statuses; +- Boolean cancel-at-period-end state; +- safe nonnegative provider timestamps with end not preceding start; +- one to 100 bounded price identifiers; +- optional bounded invoice and previously persisted source-event identifiers. + +A previously seen Stripe Customer cannot be rebound to a different ScopeWeave organization, and a previously seen Subscription cannot be rebound to a different Customer. Identity conflicts fail closed with stable conflict semantics before a new observation is accepted. + +Successful provider reads append observations rather than updating old snapshots. The local observation timestamp is monotonic per subscription even if the host wall clock moves backward; `observation_id` remains the durable append order when timestamps tie. + +## Transaction and failure semantics + +Customer identity, subscription identity, prices, the observation row, and ordered price memberships are written under one SQLite savepoint. A forced downstream junction-row failure is covered by a realistic trigger regression that proves all preceding mutations roll back together. + +A distinct cleanup-failure regression was added after working-path comparison with the calendar-subscription persistence adapter exposed the same transaction hazard: an unconditional `RELEASE` in a `finally` block can commit the outermost savepoint when `ROLLBACK TO` itself failed. SQLite documents that releasing the outermost savepoint is equivalent to commit. The observation repository now: + +- preserves the causal operation error; +- releases the savepoint only after rollback is confirmed; +- suppresses cleanup-release errors after a confirmed rollback so they cannot replace the business/persistence failure; +- leaves an unconfirmed failed savepoint open rather than risk committing partial state. + +The RED contributor head `f337b90ef1803290efb7e7df02745c7280e9d5de` added the cleanup regression and caused hosted `unit-and-api` to fail. The narrow production repair at `dada7f0ef3327fc16e0b0d02f270196285fafcd1` restored hosted `unit-and-api`, API, dependency-review, and OSV success without weakening a gate. + +## Tenant, privacy, and entitlement boundary + +The ledger stores only provider identifiers and lifecycle facts required for reconciliation. It does not store Stripe secret keys, signed raw webhook bodies, session credentials, arbitrary provider error bodies, or local entitlement decisions. + +Organization authority is purpose-bound: the local organization must already exist, the authoritative provider reader must have verified `Subscription.metadata.orgId`, and the persistence layer permanently binds the resulting Stripe identities to that tenant. A later policy layer must independently decide which observed provider state authorizes a local plan transition. + +The optional `source_event_id` is audit/reconciliation-trigger provenance only. Event arrival time is not a lifecycle ordering key and source-event presence never grants entitlement. + +## Acceptance trace + +Executable evidence includes: + +- `tests/unit/stripe-subscription-observation-ledger.test.mjs` for normalized schema shape, tenant/customer identity non-rebinding, append-only observations, validation, rollback atomicity, savepoint-cleanup failure, monotonic timestamps, source-event existence, and no direct plan mutation; +- `server/db.mjs` for bootstrap-only schema installation after the verified Stripe event ledger; +- `tests/unit/coverage-script-contract.test.mjs` for canonical coverage registration; +- `package.json` for normal unit execution and owned-production c8 instrumentation. + +PR #523 is still the repository-owned exact-contributor-head checkout-control prerequisite for merge-grade evidence across this billing stack. Repository-native successes on a stack that still inherits the prior synthetic pull-request merge checkout behavior are useful causal evidence but are not promoted to final exact-head proof. + +## Rollback and recovery + +Before protected integration, rollback removes the observation schema/bootstrap wiring, repository, focused tests, coverage registration, this doctoring record, and the corresponding active-PR changelog entry together. + +After this ledger is eventually protected-shipped, rollback must not destroy accumulated observation history merely to revert a later entitlement policy. Recovery should retain provider evidence, re-fetch authoritative Subscription state, append a new verified observation, and replay the explicitly versioned policy from a known local/provider point. + +## References + +SQLite. (n.d.). *Savepoints*. SQLite Documentation. https://sqlite.org/lang_savepoint.html + +Stripe. (n.d.). *Receive Stripe events in your webhook endpoint*. Stripe Documentation. https://docs.stripe.com/webhooks + +Stripe. (n.d.). *Retrieve a subscription*. Stripe API Reference. https://docs.stripe.com/api/subscriptions/retrieve + +Stripe. (n.d.). *Using webhooks with subscriptions*. Stripe Documentation. https://docs.stripe.com/billing/subscriptions/webhooks diff --git a/package.json b/package.json index 8e88e19c..24009c8c 100644 --- a/package.json +++ b/package.json @@ -13,9 +13,9 @@ "coverage": "npm run test:coverage", "server": "node server/server.mjs", "test:api": "node tests/api/auth-secret.test.mjs && node --env-file=tests/api/smoke.env tests/api/smoke.mjs && node tests/api/ratelimit.test.mjs && node tests/api/attachment-status.test.mjs && node tests/api/session-revocation.test.mjs && node tests/api/orchestrator-attribution.test.mjs && node tests/api/billing-checkout.test.mjs && node tests/api/billing-live-checkout.test.mjs && node tests/api/stripe-webhook.test.mjs", - "test:unit": "node tests/unit/opencode-config.test.mjs && node tests/unit/changelog-release-notes.test.mjs && node tests/unit/analytics.test.mjs && node tests/unit/cpm.test.mjs && node tests/unit/baseline-compare.test.mjs && node tests/unit/workload.test.mjs && node tests/unit/cost-evm.test.mjs && node tests/unit/msproject.test.mjs && node tests/unit/auth-password.test.mjs && node tests/unit/editor-unsaved.test.mjs && node tests/unit/static-coverage-evidence.test.mjs && node tests/unit/dep-types.test.mjs && node tests/unit/weekly-report.test.mjs && node tests/unit/clearfolio.test.mjs && node tests/unit/clearfolio-adapter-mock-hmac.test.mjs && node tests/unit/orchestrator.test.mjs && node tests/unit/orchestrator-coverage.test.mjs && node tests/unit/orchestrator-attribution.test.mjs && node tests/unit/sprint-stats.test.mjs && node tests/unit/burndown.test.mjs && node tests/unit/pm-analysis.test.mjs && node tests/unit/cloud-sync-security.test.mjs && node tests/unit/attachment-status.test.mjs && node tests/unit/clearfolio-status-signal.test.mjs && node tests/unit/coverage-script-contract.test.mjs && node tests/unit/billing-configuration.test.mjs && node tests/unit/billing-checkout-attempt.test.mjs && node tests/unit/billing-checkout-attempt-authority.test.mjs && node tests/unit/billing-checkout-reconciliation.test.mjs && node tests/unit/billing-checkout-reconciliation-authority.test.mjs && node tests/unit/billing-checkout.test.mjs && node tests/unit/billing-provider-boundary.test.mjs && node tests/unit/billing-checkout-review-regressions.test.mjs && node tests/unit/stripe-webhook-boundary.test.mjs && node tests/unit/stripe-webhook-event-ledger.test.mjs && node tests/unit/stripe-webhook-recorder-integration.test.mjs && node tests/unit/stripe-subscription-provider.test.mjs && node tests/unit/stripe-subscription-metadata-propagation.test.mjs && node tests/unit/toast-accessibility.test.mjs", - "test:coverage": "c8 --all --include=app.js --include=cloud-sync.js --include=scripts/ci/static_coverage_evidence.mjs --include=server/attachment_status.mjs --include=server/app.mjs --include=server/application_routes.mjs --include=server/auth.mjs --include=server/billing.mjs --include=server/billing_checkout_attempt.mjs --include=server/billing_configuration.mjs --include=server/clearfolio.mjs --include=server/orchestrator.mjs --include=server/stripe_webhook.mjs --include=server/stripe_webhook_event_ledger.mjs --include=server/stripe_subscription_provider.mjs --reporter=json --reporter=json-summary npm run test:coverage:cases", - "test:coverage:cases": "node tests/unit/coverage-script-contract.test.mjs && node tests/unit/attachment-status.test.mjs && node tests/unit/clearfolio-status-signal.test.mjs && node tests/unit/clearfolio-adapter-mock-hmac.test.mjs && node tests/unit/orchestrator.test.mjs && node tests/unit/orchestrator-coverage.test.mjs && node tests/unit/orchestrator-attribution.test.mjs && node tests/unit/msproject.test.mjs && node tests/unit/auth-password.test.mjs && node tests/unit/editor-unsaved.test.mjs && node tests/unit/static-coverage-evidence.test.mjs && node tests/unit/billing-configuration.test.mjs && node tests/unit/billing-checkout-attempt.test.mjs && node tests/unit/billing-checkout-attempt-authority.test.mjs && node tests/unit/billing-checkout-reconciliation.test.mjs && node tests/unit/billing-checkout-reconciliation-authority.test.mjs && node tests/unit/billing-checkout.test.mjs && node tests/unit/billing-provider-boundary.test.mjs && node tests/unit/billing-checkout-review-regressions.test.mjs && node tests/unit/stripe-webhook-boundary.test.mjs && node tests/unit/stripe-webhook-event-ledger.test.mjs && node tests/unit/stripe-webhook-recorder-integration.test.mjs && node tests/unit/stripe-subscription-provider.test.mjs && node tests/unit/stripe-subscription-metadata-propagation.test.mjs && npm run test:api", + "test:unit": "node tests/unit/opencode-config.test.mjs && node tests/unit/changelog-release-notes.test.mjs && node tests/unit/analytics.test.mjs && node tests/unit/cpm.test.mjs && node tests/unit/baseline-compare.test.mjs && node tests/unit/workload.test.mjs && node tests/unit/cost-evm.test.mjs && node tests/unit/msproject.test.mjs && node tests/unit/auth-password.test.mjs && node tests/unit/editor-unsaved.test.mjs && node tests/unit/static-coverage-evidence.test.mjs && node tests/unit/dep-types.test.mjs && node tests/unit/weekly-report.test.mjs && node tests/unit/clearfolio.test.mjs && node tests/unit/clearfolio-adapter-mock-hmac.test.mjs && node tests/unit/orchestrator.test.mjs && node tests/unit/orchestrator-coverage.test.mjs && node tests/unit/orchestrator-attribution.test.mjs && node tests/unit/sprint-stats.test.mjs && node tests/unit/burndown.test.mjs && node tests/unit/pm-analysis.test.mjs && node tests/unit/cloud-sync-security.test.mjs && node tests/unit/attachment-status.test.mjs && node tests/unit/clearfolio-status-signal.test.mjs && node tests/unit/coverage-script-contract.test.mjs && node tests/unit/billing-configuration.test.mjs && node tests/unit/billing-checkout-attempt.test.mjs && node tests/unit/billing-checkout-attempt-authority.test.mjs && node tests/unit/billing-checkout-reconciliation.test.mjs && node tests/unit/billing-checkout-reconciliation-authority.test.mjs && node tests/unit/billing-checkout.test.mjs && node tests/unit/billing-provider-boundary.test.mjs && node tests/unit/billing-checkout-review-regressions.test.mjs && node tests/unit/stripe-webhook-boundary.test.mjs && node tests/unit/stripe-webhook-event-ledger.test.mjs && node tests/unit/stripe-webhook-recorder-integration.test.mjs && node tests/unit/stripe-subscription-provider.test.mjs && node tests/unit/stripe-subscription-metadata-propagation.test.mjs && node tests/unit/stripe-subscription-observation-ledger.test.mjs && node tests/unit/toast-accessibility.test.mjs", + "test:coverage": "c8 --all --include=app.js --include=cloud-sync.js --include=scripts/ci/static_coverage_evidence.mjs --include=server/attachment_status.mjs --include=server/app.mjs --include=server/application_routes.mjs --include=server/auth.mjs --include=server/billing.mjs --include=server/billing_checkout_attempt.mjs --include=server/billing_configuration.mjs --include=server/clearfolio.mjs --include=server/orchestrator.mjs --include=server/stripe_webhook.mjs --include=server/stripe_webhook_event_ledger.mjs --include=server/stripe_subscription_provider.mjs --include=server/stripe_subscription_observation_ledger.mjs --reporter=json --reporter=json-summary npm run test:coverage:cases", + "test:coverage:cases": "node tests/unit/coverage-script-contract.test.mjs && node tests/unit/attachment-status.test.mjs && node tests/unit/clearfolio-status-signal.test.mjs && node tests/unit/clearfolio-adapter-mock-hmac.test.mjs && node tests/unit/orchestrator.test.mjs && node tests/unit/orchestrator-coverage.test.mjs && node tests/unit/orchestrator-attribution.test.mjs && node tests/unit/msproject.test.mjs && node tests/unit/auth-password.test.mjs && node tests/unit/editor-unsaved.test.mjs && node tests/unit/static-coverage-evidence.test.mjs && node tests/unit/billing-configuration.test.mjs && node tests/unit/billing-checkout-attempt.test.mjs && node tests/unit/billing-checkout-attempt-authority.test.mjs && node tests/unit/billing-checkout-reconciliation.test.mjs && node tests/unit/billing-checkout-reconciliation-authority.test.mjs && node tests/unit/billing-checkout.test.mjs && node tests/unit/billing-provider-boundary.test.mjs && node tests/unit/billing-checkout-review-regressions.test.mjs && node tests/unit/stripe-webhook-boundary.test.mjs && node tests/unit/stripe-webhook-event-ledger.test.mjs && node tests/unit/stripe-webhook-recorder-integration.test.mjs && node tests/unit/stripe-subscription-provider.test.mjs && node tests/unit/stripe-subscription-metadata-propagation.test.mjs && node tests/unit/stripe-subscription-observation-ledger.test.mjs && npm run test:api", "test:e2e": "playwright test", "test:e2e:headed": "playwright test --headed", "test:e2e:cloud": "playwright install chromium && playwright test tests/e2e/cloud.spec.js tests/e2e/toast-accessibility.spec.js", diff --git a/server/db.mjs b/server/db.mjs index 7e90b51f..ff0bcb5e 100644 --- a/server/db.mjs +++ b/server/db.mjs @@ -13,6 +13,10 @@ import { createSqliteStripeWebhookEventRepository, installStripeWebhookEventSchema, } from './stripe_webhook_event_ledger.mjs'; +import { + createSqliteStripeSubscriptionObservationRepository, + installStripeSubscriptionObservationSchema, +} from './stripe_subscription_observation_ledger.mjs'; const __dirname = dirname(fileURLToPath(import.meta.url)); const dbPath = process.env.SCOPEWEAVE_DB || join(__dirname, '..', 'data.db'); @@ -192,6 +196,8 @@ export const billingCheckoutAttempts = createSqliteBillingCheckoutAttemptReposit installStripeWebhookEventSchema(db); export const stripeWebhookEvents = createSqliteStripeWebhookEventRepository(db); configureStripeWebhookEventRecorder((evidence) => stripeWebhookEvents.recordVerifiedEvent(evidence)); +installStripeSubscriptionObservationSchema(db); +export const stripeSubscriptionObservations = createSqliteStripeSubscriptionObservationRepository(db); // node:sqlite returns lastInsertRowid as number|bigint; normalize to Number. export const rowid = (r) => Number(r.lastInsertRowid); diff --git a/server/stripe_subscription_observation_ledger.mjs b/server/stripe_subscription_observation_ledger.mjs new file mode 100644 index 00000000..491685d6 --- /dev/null +++ b/server/stripe_subscription_observation_ledger.mjs @@ -0,0 +1,338 @@ +const MAX_PROVIDER_ID_LENGTH = 255; +const MAX_SUBSCRIPTION_ITEMS = 100; +const SAVEPOINT_NAME = 'billing_stripe_subscription_observation_write'; +const PROVIDER_IDENTIFIER_PATTERN = /^[A-Za-z0-9_:-]+$/u; +const STRIPE_SUBSCRIPTION_STATUSES = new Set([ + 'incomplete', + 'incomplete_expired', + 'trialing', + 'active', + 'past_due', + 'canceled', + 'unpaid', + 'paused', +]); + +/** Stable fail-closed persistence error for authoritative Stripe observations. */ +export class StripeSubscriptionObservationError extends Error { + /** + * @param {string} code stable machine-readable failure code + * @param {number} status HTTP status suitable for a future service adapter + */ + constructor(code, status = 400) { + super(code); + this.name = 'StripeSubscriptionObservationError'; + this.code = code; + this.status = status; + } +} + +function observationError(code = 'stripe_subscription_observation_invalid', status = 400) { + return new StripeSubscriptionObservationError(code, status); +} + +function requiredIdentifier(value) { + if (typeof value !== 'string' + || value.length === 0 + || value.length > MAX_PROVIDER_ID_LENGTH + || !PROVIDER_IDENTIFIER_PATTERN.test(value)) { + throw observationError(); + } + return value; +} + +function positiveOrganizationId(value) { + const parsed = Number(value); + if (!Number.isSafeInteger(parsed) || parsed <= 0) throw observationError(); + return parsed; +} + +function nonNegativeInteger(value) { + if (!Number.isSafeInteger(value) || value < 0) throw observationError(); + return value; +} + +function nullableTimestamp(value) { + if (value == null) return null; + return nonNegativeInteger(value); +} + +function nullableIdentifier(value) { + if (value == null) return null; + return requiredIdentifier(value); +} + +function sourceEventIdentifier(value) { + if (value == null) return null; + return requiredIdentifier(value); +} + +function safeNow(now) { + const value = Number(now()); + if (!Number.isSafeInteger(value) || value < 0) throw observationError(); + return value; +} + +function normalizedSnapshot(snapshot) { + if (!snapshot || typeof snapshot !== 'object' || Array.isArray(snapshot)) { + throw observationError(); + } + + const organizationId = positiveOrganizationId(snapshot.organizationId); + const subscriptionId = requiredIdentifier(snapshot.subscriptionId); + const customerId = requiredIdentifier(snapshot.customerId); + if (!STRIPE_SUBSCRIPTION_STATUSES.has(snapshot.status)) throw observationError(); + if (typeof snapshot.cancelAtPeriodEnd !== 'boolean') throw observationError(); + + const currentPeriodStartSec = nonNegativeInteger(snapshot.currentPeriodStartSec); + const currentPeriodEndSec = nonNegativeInteger(snapshot.currentPeriodEndSec); + if (currentPeriodEndSec < currentPeriodStartSec) throw observationError(); + + if (!Array.isArray(snapshot.priceIds) + || snapshot.priceIds.length === 0 + || snapshot.priceIds.length > MAX_SUBSCRIPTION_ITEMS) { + throw observationError(); + } + const priceIds = snapshot.priceIds.map(requiredIdentifier); + + return { + organizationId, + subscriptionId, + customerId, + status: snapshot.status, + cancelAtPeriodEnd: snapshot.cancelAtPeriodEnd, + currentPeriodStartSec, + currentPeriodEndSec, + canceledAtSec: nullableTimestamp(snapshot.canceledAtSec), + endedAtSec: nullableTimestamp(snapshot.endedAtSec), + trialEndSec: nullableTimestamp(snapshot.trialEndSec), + latestInvoiceId: nullableIdentifier(snapshot.latestInvoiceId), + priceIds, + }; +} + +function withSavepoint(database, operation) { + database.exec(`SAVEPOINT ${SAVEPOINT_NAME}`); + try { + const result = operation(); + database.exec(`RELEASE SAVEPOINT ${SAVEPOINT_NAME}`); + return result; + } catch (error) { + let rollbackSucceeded = false; + try { + database.exec(`ROLLBACK TO SAVEPOINT ${SAVEPOINT_NAME}`); + rollbackSucceeded = true; + } catch { + // An unconfirmed rollback must keep the savepoint open rather than risk committing failed state. + } + if (rollbackSucceeded) { + try { + database.exec(`RELEASE SAVEPOINT ${SAVEPOINT_NAME}`); + } catch { + // Cleanup failure after a confirmed rollback must not replace the causal operation error. + } + } + throw error; + } +} + +/** + * Install normalized provider-identity and authoritative-observation relations. + * + * Customer, Subscription, and Price identifiers are stored once. Every provider + * read is then appended as a separate immutable observation, with its ordered + * price membership stored in a junction relation. The observation deliberately + * does not contain organization plan or entitlement state: a separate policy + * layer must decide whether provider facts authorize a local transition. + * + * This schema must be installed after `billing_stripe_webhook_events` when source + * event provenance is available. Installation belongs to bootstrap/migrations, + * never a request handler. + * + * @param {import('node:sqlite').DatabaseSync} database bootstrapped database + * @returns {void} + */ +export function installStripeSubscriptionObservationSchema(database) { + database.exec(` + CREATE TABLE IF NOT EXISTS billing_stripe_customers ( + customer_id TEXT PRIMARY KEY CHECK(length(customer_id) BETWEEN 1 AND ${MAX_PROVIDER_ID_LENGTH}), + organization_id INTEGER NOT NULL REFERENCES orgs(id) ON DELETE CASCADE, + first_observed_at_ms INTEGER NOT NULL CHECK(first_observed_at_ms >= 0) + ); + CREATE INDEX IF NOT EXISTS billing_stripe_customer_organizations + ON billing_stripe_customers(organization_id, customer_id); + + CREATE TABLE IF NOT EXISTS billing_stripe_subscriptions ( + subscription_id TEXT PRIMARY KEY CHECK(length(subscription_id) BETWEEN 1 AND ${MAX_PROVIDER_ID_LENGTH}), + customer_id TEXT NOT NULL REFERENCES billing_stripe_customers(customer_id) ON DELETE CASCADE, + first_observed_at_ms INTEGER NOT NULL CHECK(first_observed_at_ms >= 0) + ); + CREATE INDEX IF NOT EXISTS billing_stripe_customer_subscriptions + ON billing_stripe_subscriptions(customer_id, subscription_id); + + CREATE TABLE IF NOT EXISTS billing_stripe_prices ( + price_id TEXT PRIMARY KEY CHECK(length(price_id) BETWEEN 1 AND ${MAX_PROVIDER_ID_LENGTH}), + first_observed_at_ms INTEGER NOT NULL CHECK(first_observed_at_ms >= 0) + ); + + CREATE TABLE IF NOT EXISTS billing_stripe_subscription_observations ( + observation_id INTEGER PRIMARY KEY, + subscription_id TEXT NOT NULL REFERENCES billing_stripe_subscriptions(subscription_id) ON DELETE CASCADE, + source_event_id TEXT REFERENCES billing_stripe_webhook_events(event_id) ON DELETE RESTRICT, + observed_at_ms INTEGER NOT NULL CHECK(observed_at_ms >= 0), + subscription_status TEXT NOT NULL CHECK(subscription_status IN ( + 'incomplete','incomplete_expired','trialing','active','past_due','canceled','unpaid','paused' + )), + cancel_at_period_end INTEGER NOT NULL CHECK(cancel_at_period_end IN (0,1)), + current_period_start_sec INTEGER NOT NULL CHECK(current_period_start_sec >= 0), + current_period_end_sec INTEGER NOT NULL CHECK(current_period_end_sec >= current_period_start_sec), + canceled_at_sec INTEGER CHECK(canceled_at_sec IS NULL OR canceled_at_sec >= 0), + ended_at_sec INTEGER CHECK(ended_at_sec IS NULL OR ended_at_sec >= 0), + trial_end_sec INTEGER CHECK(trial_end_sec IS NULL OR trial_end_sec >= 0), + latest_invoice_id TEXT CHECK(latest_invoice_id IS NULL OR length(latest_invoice_id) BETWEEN 1 AND ${MAX_PROVIDER_ID_LENGTH}) + ); + CREATE INDEX IF NOT EXISTS billing_stripe_subscription_observation_history + ON billing_stripe_subscription_observations(subscription_id, observed_at_ms, observation_id); + CREATE INDEX IF NOT EXISTS billing_stripe_source_event_observations + ON billing_stripe_subscription_observations(source_event_id, observation_id); + + CREATE TABLE IF NOT EXISTS billing_stripe_subscription_observation_prices ( + observation_id INTEGER NOT NULL REFERENCES billing_stripe_subscription_observations(observation_id) ON DELETE CASCADE, + position_index INTEGER NOT NULL CHECK(position_index >= 0), + price_id TEXT NOT NULL REFERENCES billing_stripe_prices(price_id) ON DELETE RESTRICT, + PRIMARY KEY(observation_id, position_index) + ); + CREATE INDEX IF NOT EXISTS billing_stripe_price_observations + ON billing_stripe_subscription_observation_prices(price_id, observation_id); + `); +} + +/** + * Create the SQLite persistence port for authoritative Stripe Subscription reads. + * + * A provider identifier is permanently bound to the first tenant/customer seen; + * later attempts to rebind the same Stripe Customer or Subscription fail closed. + * Successful reads append observations instead of overwriting previous evidence. + * A caller may link an observation to a previously verified webhook event, but + * that provenance never grants entitlement and is never used as an ordering key. + * + * @param {import('node:sqlite').DatabaseSync} database bootstrapped database + * @param {object} [dependencies] deterministic dependency seams + * @param {() => number} [dependencies.now] wall-clock milliseconds + * @returns {{recordAuthoritativeObservation(input: {snapshot: Record, sourceEventId?: string|null}): Readonly<{observationId: number, subscriptionId: string, observedAtMs: number}>}} + */ +export function createSqliteStripeSubscriptionObservationRepository(database, { now = Date.now } = {}) { + if (!database || typeof database.prepare !== 'function' || typeof database.exec !== 'function') { + throw new TypeError('database must provide SQLite prepare/exec operations'); + } + if (typeof now !== 'function') throw new TypeError('now must be a function'); + + const selectOrganization = database.prepare('SELECT id FROM orgs WHERE id = ?'); + const selectSourceEvent = database.prepare( + 'SELECT event_id FROM billing_stripe_webhook_events WHERE event_id = ?', + ); + const selectCustomer = database.prepare( + 'SELECT organization_id FROM billing_stripe_customers WHERE customer_id = ?', + ); + const insertCustomer = database.prepare(` + INSERT INTO billing_stripe_customers(customer_id, organization_id, first_observed_at_ms) + VALUES(?,?,?) + `); + const selectSubscription = database.prepare( + 'SELECT customer_id FROM billing_stripe_subscriptions WHERE subscription_id = ?', + ); + const insertSubscription = database.prepare(` + INSERT INTO billing_stripe_subscriptions(subscription_id, customer_id, first_observed_at_ms) + VALUES(?,?,?) + `); + const selectLastObserved = database.prepare(` + SELECT MAX(observed_at_ms) AS observed_at_ms + FROM billing_stripe_subscription_observations + WHERE subscription_id = ? + `); + const insertPrice = database.prepare(` + INSERT OR IGNORE INTO billing_stripe_prices(price_id, first_observed_at_ms) + VALUES(?,?) + `); + const insertObservation = database.prepare(` + INSERT INTO billing_stripe_subscription_observations( + subscription_id, source_event_id, observed_at_ms, subscription_status, + cancel_at_period_end, current_period_start_sec, current_period_end_sec, + canceled_at_sec, ended_at_sec, trial_end_sec, latest_invoice_id + ) VALUES(?,?,?,?,?,?,?,?,?,?,?) + `); + const insertObservationPrice = database.prepare(` + INSERT INTO billing_stripe_subscription_observation_prices( + observation_id, position_index, price_id + ) VALUES(?,?,?) + `); + + return { + /** Append one validated provider snapshot without making an entitlement decision. */ + recordAuthoritativeObservation({ snapshot, sourceEventId = null }) { + const normalized = normalizedSnapshot(snapshot); + const sourceEvent = sourceEventIdentifier(sourceEventId); + const clockMs = safeNow(now); + + return withSavepoint(database, () => { + if (!selectOrganization.get(normalized.organizationId)) { + throw observationError('stripe_subscription_observation_invalid'); + } + if (sourceEvent && !selectSourceEvent.get(sourceEvent)) { + throw observationError('stripe_subscription_source_event_unknown', 409); + } + + const existingCustomer = selectCustomer.get(normalized.customerId); + if (existingCustomer) { + if (Number(existingCustomer.organization_id) !== normalized.organizationId) { + throw observationError('stripe_subscription_identity_conflict', 409); + } + } else { + insertCustomer.run(normalized.customerId, normalized.organizationId, clockMs); + } + + const existingSubscription = selectSubscription.get(normalized.subscriptionId); + if (existingSubscription) { + if (existingSubscription.customer_id !== normalized.customerId) { + throw observationError('stripe_subscription_identity_conflict', 409); + } + } else { + insertSubscription.run(normalized.subscriptionId, normalized.customerId, clockMs); + } + + const priorObserved = selectLastObserved.get(normalized.subscriptionId)?.observed_at_ms; + const observedAtMs = Number.isSafeInteger(priorObserved) + ? Math.max(clockMs, priorObserved) + : clockMs; + + for (const priceId of normalized.priceIds) { + insertPrice.run(priceId, observedAtMs); + } + + const observation = insertObservation.run( + normalized.subscriptionId, + sourceEvent, + observedAtMs, + normalized.status, + normalized.cancelAtPeriodEnd ? 1 : 0, + normalized.currentPeriodStartSec, + normalized.currentPeriodEndSec, + normalized.canceledAtSec, + normalized.endedAtSec, + normalized.trialEndSec, + normalized.latestInvoiceId, + ); + const observationId = Number(observation.lastInsertRowid); + normalized.priceIds.forEach((priceId, positionIndex) => { + insertObservationPrice.run(observationId, positionIndex, priceId); + }); + + return Object.freeze({ + observationId, + subscriptionId: normalized.subscriptionId, + observedAtMs, + }); + }); + }, + }; +} diff --git a/tests/unit/coverage-script-contract.test.mjs b/tests/unit/coverage-script-contract.test.mjs index bd17c057..bb1e34d0 100644 --- a/tests/unit/coverage-script-contract.test.mjs +++ b/tests/unit/coverage-script-contract.test.mjs @@ -64,6 +64,11 @@ assert.match( /--include=server\/stripe_subscription_provider\.mjs/, 'the authoritative Stripe subscription reader is instrumented', ); +assert.match( + scripts['test:coverage'], + /--include=server\/stripe_subscription_observation_ledger\.mjs/, + 'the authoritative Stripe subscription observation ledger is instrumented', +); assert.match( scripts['test:coverage:cases'], /tests\/unit\/clearfolio-status-signal\.test\.mjs/, @@ -129,6 +134,16 @@ assert.match( /tests\/unit\/stripe-subscription-metadata-propagation\.test\.mjs/, 'the subscription tenant-metadata propagation regression executes under c8', ); +assert.match( + scripts['test:coverage:cases'], + /tests\/unit\/stripe-subscription-observation-ledger\.test\.mjs/, + 'the authoritative Stripe subscription observation regression executes under c8', +); +assert.match( + scripts['test:unit'], + /tests\/unit\/stripe-subscription-observation-ledger\.test\.mjs/, + 'normal unit CI executes the authoritative Stripe subscription observation regression', +); assert.doesNotMatch( scripts['test:coverage:cases'], /npm run (?:coverage|test:coverage)(?:\s|$)/, diff --git a/tests/unit/stripe-subscription-observation-ledger.test.mjs b/tests/unit/stripe-subscription-observation-ledger.test.mjs new file mode 100644 index 00000000..14cc235d --- /dev/null +++ b/tests/unit/stripe-subscription-observation-ledger.test.mjs @@ -0,0 +1,316 @@ +import assert from 'node:assert/strict'; +import { test } from 'node:test'; +import { DatabaseSync } from 'node:sqlite'; +import { + StripeSubscriptionObservationError, + createSqliteStripeSubscriptionObservationRepository, + installStripeSubscriptionObservationSchema, +} from '../../server/stripe_subscription_observation_ledger.mjs'; + +function createDatabase() { + const database = new DatabaseSync(':memory:'); + database.exec('PRAGMA foreign_keys = ON'); + database.exec(` + CREATE TABLE users ( + id INTEGER PRIMARY KEY, + email TEXT NOT NULL UNIQUE, + password_hash TEXT NOT NULL, + name TEXT NOT NULL DEFAULT '' + ); + CREATE TABLE orgs ( + id INTEGER PRIMARY KEY, + name TEXT NOT NULL, + owner_id INTEGER NOT NULL REFERENCES users(id), + plan TEXT NOT NULL DEFAULT 'free' + ); + CREATE TABLE billing_stripe_webhook_events ( + event_id TEXT PRIMARY KEY + ); + INSERT INTO users(id, email, password_hash, name) + VALUES(1, 'owner@example.test', 'hash', 'Owner'); + INSERT INTO orgs(id, name, owner_id, plan) + VALUES(42, 'Acquisition-grade buyer', 1, 'free'), + (84, 'Other tenant', 1, 'free'); + INSERT INTO billing_stripe_webhook_events(event_id) + VALUES('evt_subscription_reconcile_1'); + `); + installStripeSubscriptionObservationSchema(database); + return database; +} + +function snapshot(overrides = {}) { + return Object.freeze({ + subscriptionId: 'sub_scopeweave_42', + customerId: 'cus_scopeweave_42', + organizationId: 42, + status: 'active', + cancelAtPeriodEnd: false, + currentPeriodStartSec: 1_787_000_000, + currentPeriodEndSec: 1_789_678_400, + canceledAtSec: null, + endedAtSec: null, + trialEndSec: null, + latestInvoiceId: 'in_scopeweave_42', + priceIds: Object.freeze(['price_scopeweave_pro', 'price_scopeweave_storage']), + ...overrides, + }); +} + +function setup(now = () => 1_787_000_100_000) { + const database = createDatabase(); + const repository = createSqliteStripeSubscriptionObservationRepository(database, { now }); + return { database, repository }; +} + +function tableColumns(database, tableName) { + return database.prepare(`PRAGMA table_info('${tableName}')`).all().map((row) => row.name); +} + +test('bootstrap schema keeps customer, subscription, observation, and price facts in normalized relations', () => { + const database = createDatabase(); + installStripeSubscriptionObservationSchema(database); + + assert.deepEqual(tableColumns(database, 'billing_stripe_customers'), [ + 'customer_id', 'organization_id', 'first_observed_at_ms', + ]); + assert.deepEqual(tableColumns(database, 'billing_stripe_subscriptions'), [ + 'subscription_id', 'customer_id', 'first_observed_at_ms', + ]); + assert.deepEqual(tableColumns(database, 'billing_stripe_prices'), [ + 'price_id', 'first_observed_at_ms', + ]); + assert.deepEqual(tableColumns(database, 'billing_stripe_subscription_observations'), [ + 'observation_id', 'subscription_id', 'source_event_id', 'observed_at_ms', + 'subscription_status', 'cancel_at_period_end', 'current_period_start_sec', + 'current_period_end_sec', 'canceled_at_sec', 'ended_at_sec', 'trial_end_sec', + 'latest_invoice_id', + ]); + assert.deepEqual(tableColumns(database, 'billing_stripe_subscription_observation_prices'), [ + 'observation_id', 'position_index', 'price_id', + ]); + + const ownedTables = database.prepare(` + SELECT name FROM sqlite_master + WHERE type = 'table' AND name LIKE 'billing_stripe_%' + ORDER BY name + `).all().map((row) => row.name); + for (const name of ownedTables.filter((name) => name !== 'billing_stripe_webhook_events')) { + assert.match(name, /^[a-z]+_[a-z0-9]+(?:_[a-z0-9]+)+$/); + } +}); + +test('one authoritative snapshot records tenant-bound identity and an immutable observation without changing entitlement', () => { + const { database, repository } = setup(); + const result = repository.recordAuthoritativeObservation({ + snapshot: snapshot(), + sourceEventId: 'evt_subscription_reconcile_1', + }); + + assert.deepEqual(result, { + observationId: 1, + subscriptionId: 'sub_scopeweave_42', + observedAtMs: 1_787_000_100_000, + }); + assert.deepEqual({ ...database.prepare('SELECT * FROM billing_stripe_customers').get() }, { + customer_id: 'cus_scopeweave_42', + organization_id: 42, + first_observed_at_ms: 1_787_000_100_000, + }); + assert.deepEqual({ ...database.prepare('SELECT * FROM billing_stripe_subscriptions').get() }, { + subscription_id: 'sub_scopeweave_42', + customer_id: 'cus_scopeweave_42', + first_observed_at_ms: 1_787_000_100_000, + }); + assert.deepEqual({ ...database.prepare('SELECT * FROM billing_stripe_subscription_observations').get() }, { + observation_id: 1, + subscription_id: 'sub_scopeweave_42', + source_event_id: 'evt_subscription_reconcile_1', + observed_at_ms: 1_787_000_100_000, + subscription_status: 'active', + cancel_at_period_end: 0, + current_period_start_sec: 1_787_000_000, + current_period_end_sec: 1_789_678_400, + canceled_at_sec: null, + ended_at_sec: null, + trial_end_sec: null, + latest_invoice_id: 'in_scopeweave_42', + }); + assert.deepEqual( + database.prepare(` + SELECT position_index, price_id + FROM billing_stripe_subscription_observation_prices + ORDER BY position_index + `).all().map((row) => ({ ...row })), + [ + { position_index: 0, price_id: 'price_scopeweave_pro' }, + { position_index: 1, price_id: 'price_scopeweave_storage' }, + ], + ); + assert.equal(database.prepare('SELECT plan FROM orgs WHERE id = 42').get().plan, 'free'); +}); + +test('repeat authoritative reads append evidence while reusing normalized provider identities', () => { + let clock = 1_787_000_100_000; + const { database, repository } = setup(() => clock); + repository.recordAuthoritativeObservation({ snapshot: snapshot() }); + clock += 25; + repository.recordAuthoritativeObservation({ + snapshot: snapshot({ status: 'past_due', cancelAtPeriodEnd: true }), + }); + + assert.equal(database.prepare('SELECT COUNT(*) AS count FROM billing_stripe_customers').get().count, 1); + assert.equal(database.prepare('SELECT COUNT(*) AS count FROM billing_stripe_subscriptions').get().count, 1); + assert.equal(database.prepare('SELECT COUNT(*) AS count FROM billing_stripe_prices').get().count, 2); + assert.deepEqual( + database.prepare(` + SELECT observed_at_ms, subscription_status, cancel_at_period_end + FROM billing_stripe_subscription_observations + ORDER BY observation_id + `).all().map((row) => ({ ...row })), + [ + { observed_at_ms: 1_787_000_100_000, subscription_status: 'active', cancel_at_period_end: 0 }, + { observed_at_ms: 1_787_000_100_025, subscription_status: 'past_due', cancel_at_period_end: 1 }, + ], + ); +}); + +test('provider customer and subscription identifiers can never be rebound across tenants or customers', () => { + const { database, repository } = setup(); + repository.recordAuthoritativeObservation({ snapshot: snapshot() }); + + assert.throws( + () => repository.recordAuthoritativeObservation({ + snapshot: snapshot({ organizationId: 84 }), + }), + (error) => error instanceof StripeSubscriptionObservationError + && error.code === 'stripe_subscription_identity_conflict', + ); + assert.throws( + () => repository.recordAuthoritativeObservation({ + snapshot: snapshot({ customerId: 'cus_scopeweave_other' }), + }), + (error) => error instanceof StripeSubscriptionObservationError + && error.code === 'stripe_subscription_identity_conflict', + ); + + assert.equal(database.prepare('SELECT COUNT(*) AS count FROM billing_stripe_subscription_observations').get().count, 1); + assert.equal(database.prepare('SELECT COUNT(*) AS count FROM billing_stripe_customers').get().count, 1); +}); + +test('invalid snapshots and unknown source-event evidence fail before durable state changes', () => { + const invalidSnapshots = [ + snapshot({ organizationId: 0 }), + snapshot({ organizationId: 1.5 }), + snapshot({ subscriptionId: '' }), + snapshot({ customerId: {} }), + snapshot({ status: 'mystery' }), + snapshot({ cancelAtPeriodEnd: 'false' }), + snapshot({ currentPeriodStartSec: -1 }), + snapshot({ currentPeriodEndSec: 1_786_999_999 }), + snapshot({ priceIds: [] }), + snapshot({ priceIds: [''] }), + snapshot({ latestInvoiceId: {} }), + ]; + + for (const candidate of invalidSnapshots) { + const { database, repository } = setup(); + assert.throws( + () => repository.recordAuthoritativeObservation({ snapshot: candidate }), + (error) => error instanceof StripeSubscriptionObservationError + && error.code === 'stripe_subscription_observation_invalid', + ); + assert.equal(database.prepare('SELECT COUNT(*) AS count FROM billing_stripe_subscription_observations').get().count, 0); + } + + const { database, repository } = setup(); + assert.throws( + () => repository.recordAuthoritativeObservation({ + snapshot: snapshot(), + sourceEventId: 'evt_not_persisted', + }), + (error) => error instanceof StripeSubscriptionObservationError + && error.code === 'stripe_subscription_source_event_unknown', + ); + assert.equal(database.prepare('SELECT COUNT(*) AS count FROM billing_stripe_customers').get().count, 0); +}); + +test('a downstream observation-price write failure rolls back every identity and observation mutation', () => { + const { database, repository } = setup(); + database.exec(` + CREATE TRIGGER billing_stripe_test_price_failure + BEFORE INSERT ON billing_stripe_subscription_observation_prices + WHEN NEW.position_index = 1 + BEGIN + SELECT RAISE(ABORT, 'simulated downstream persistence failure'); + END; + `); + + assert.throws( + () => repository.recordAuthoritativeObservation({ snapshot: snapshot() }), + /simulated downstream persistence failure/, + ); + + for (const table of [ + 'billing_stripe_customers', + 'billing_stripe_subscriptions', + 'billing_stripe_prices', + 'billing_stripe_subscription_observations', + 'billing_stripe_subscription_observation_prices', + ]) { + assert.equal(database.prepare(`SELECT COUNT(*) AS count FROM ${table}`).get().count, 0); + } + assert.equal(database.prepare('SELECT plan FROM orgs WHERE id = 42').get().plan, 'free'); +}); + +test('savepoint rollback cleanup preserves the causal write failure and never releases unconfirmed state', () => { + const database = createDatabase(); + database.exec(` + CREATE TRIGGER billing_stripe_test_price_failure + BEFORE INSERT ON billing_stripe_subscription_observation_prices + WHEN NEW.position_index = 1 + BEGIN + SELECT RAISE(ABORT, 'causal observation write failure'); + END; + `); + + const executed = []; + const guardedDatabase = { + prepare: database.prepare.bind(database), + exec(sql) { + executed.push(sql); + if (sql === 'ROLLBACK TO SAVEPOINT billing_stripe_subscription_observation_write') { + throw new Error('simulated rollback cleanup failure'); + } + return database.exec(sql); + }, + }; + const repository = createSqliteStripeSubscriptionObservationRepository(guardedDatabase); + + assert.throws( + () => repository.recordAuthoritativeObservation({ snapshot: snapshot() }), + /causal observation write failure/, + ); + assert.equal( + executed.filter((sql) => sql === 'RELEASE SAVEPOINT billing_stripe_subscription_observation_write').length, + 0, + 'failed rollback must not release an unconfirmed savepoint and accidentally commit partial state', + ); +}); + +test('observation timestamps are monotonic per subscription despite local clock rollback', () => { + const times = [1_787_000_100_000, 1_787_000_099_000]; + const { database, repository } = setup(() => times.shift()); + repository.recordAuthoritativeObservation({ snapshot: snapshot() }); + const result = repository.recordAuthoritativeObservation({ + snapshot: snapshot({ status: 'past_due' }), + }); + + assert.equal(result.observedAtMs, 1_787_000_100_000); + assert.deepEqual( + database.prepare(` + SELECT observed_at_ms FROM billing_stripe_subscription_observations + ORDER BY observation_id + `).all().map((row) => row.observed_at_ms), + [1_787_000_100_000, 1_787_000_100_000], + ); +});