Enforce storage bytes entitlement

[kentcdodds]

Summary

• Added a built-in storage_bytes D1 byte estimator for countable user-owned durable payloads (email raw/body metadata, values, encrypted secrets, memories, saved package/job/runtime metadata, and related projections).
• Wired storage quota enforcement into email storage, values, secrets, memories, saved-package projections, StorageRunner write tools, direct storage_query writable calls, and package app storage RPC writes.
• Tightened write estimates after AI review: upserts reserve only net-positive byte growth, writable SQL estimates query/params, and inbound email reserves raw MIME plus extracted/stored metadata.
• Updated entitlement architecture docs to describe counted storage, StorageRunner getCurrent handling, and current gaps for stores without reliable byte metadata (Artifacts/KV bodies/Vectorize scans).
• Added focused tests for planned users at the storage limit, NULL-plan users remaining unaffected, inbound email storage-byte rejection, StorageRunner write chokepoint enforcement, and net-positive upsert byte deltas.

Verification

• npm run test -- packages/worker/src/package-registry/service.node.test.ts packages/worker/src/entitlements/entitlements.node.test.ts packages/worker/src/storage-runner.workers.test.ts packages/worker/src/email/inbound-entitlements.workers.test.ts
• npm run typecheck
• npm run format:check
• npm run validate (format, lint, typecheck, unit tests, Playwright E2E, and MCP E2E all exited 0)
• GitHub PR checks on latest head: Validate passed, preview passed, Cursor Bugbot passed, CodeRabbit passed

System recap — extends existing primitives (medium risk)

Mode: recap · Base: main @ 80e35c5 · Head: a57a8ed

Classification: extends — this PR changes entitlement enforcement behavior for an existing plan resource and wires existing storage primitives into that guard.

Primitives touched

Primitive	Group	Impact
entitlements	auth	extends — storage_bytes now has a D1 byte counter, shared storage-byte assertion path, and byte-estimation/delta helpers
durable-storage	assistant	extends — StorageRunner write tools and package app storage RPCs check storage quota before writes
d1-app-db	storage	composes — existing user-scoped tables are read for byte estimates; no schema change
email	assistant	composes — inbound/outbound message storage checks storage quota
values	assistant	composes — value writes pass estimated payload byte deltas to the entitlement guard
secrets	assistant	composes — encrypted secret writes pass estimated stored byte deltas to the entitlement guard
memories	assistant	composes — memory upserts pass estimated stored byte deltas to the entitlement guard
saved-packages	assistant	composes — saved package projection writes pass estimated stored byte deltas to the entitlement guard
mcp-server	surfaces	composes — capability handlers thread caller email to storage enforcement

System map

flowchart LR
	mcpServer["mcp-server"]:::touched --> entitlements["entitlements"]:::extended
	appUi["app-ui"]:::untouched --> entitlements
	entitlements --> d1AppDb["d1-app-db"]:::touched
	entitlements --> durableStorage["durable-storage"]:::extended
	email["email"]:::touched --> entitlements
	values["values"]:::touched --> entitlements
	secrets["secrets"]:::touched --> entitlements
	memories["memories"]:::touched --> entitlements
	savedPackages["saved-packages"]:::touched --> entitlements
	packageApps["package-apps"]:::touched --> durableStorage
	classDef touched fill:#1a7f37,color:#fff
	classDef extended fill:#9a6700,color:#fff
	classDef added fill:#cf222e,color:#fff
	classDef untouched fill:#57606a,color:#fff

Loading

Change flow

sequenceDiagram
	participant Writer as Storage/write chokepoint
	participant Ent as assertWithinStorageBytesEntitlement
	participant D1 as APP_DB byte counter
	participant DO as StorageRunner bucket estimate
	Writer->>Ent: userId, email, requested byte delta
	Ent->>D1: resolve plan by verified user email
	alt known plan
		Ent->>D1: read user-scoped D1 byte estimate
		opt StorageRunner write
			Ent->>DO: read target bucket estimatedBytes
		end
		Ent-->>Writer: allow or EntitlementLimitError(storage_bytes)
	else NULL/unknown plan
		Ent-->>Writer: allow without counting
	end

Loading

Invariants

• per-user-isolation: all byte-counting SQL filters by user_id, transitive child counts join back to user-owned parent rows, and StorageRunner estimates are read from the (userId, storageId) Durable Object namespace.
• NULL-plan invariant is preserved for storage_bytes: users without a known plan return before byte counting and remain unlimited for this resource.

  

Summary by CodeRabbit

• New Features

  • Added storage-byte quota enforcement across email, value, secret, memory, package, and storage-write flows.
  • Expanded storage write checks to account for estimated item size before saving data.
  • Improved handling of user email context for saved content and connected integrations.

• Bug Fixes

  • Inbound and outbound email actions now reject oversized storage usage earlier and more consistently.
  • Storage runner write tools now block writes when a user has reached their storage limit.

• Documentation

  • Updated entitlement docs to describe storage-byte limits and where they are enforced.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR adds a storage_bytes entitlement resource: a D1-based byte-usage estimator, enforcement wrappers, and checks wired into StorageRunner writes, inbound/outbound email, values/secrets/memory upserts, and package-registry projections, plus userEmail propagation through several save/upsert call sites and corresponding tests/docs.

Changes

Storage-bytes entitlement enforcement

Layer / File(s)	Summary
Storage-bytes usage estimator and entitlement service docs/contributing/architecture/entitlements.md, packages/worker/src/entitlements/service.ts, packages/worker/src/entitlements/entitlements.node.test.ts	Documents and implements readUserD1StorageBytes, byte-estimation helpers, and assertWithinStorageBytesEntitlement; wires storage_bytes into usage-reading logic and updates enforcement-point docs and unit tests.
StorageRunner write entitlement wiring packages/worker/src/storage-runner.ts, packages/worker/src/storage-runner.workers.test.ts, packages/worker/src/mcp/capabilities/storage/storage-query.ts, packages/worker/src/package-runtime/package-app.ts, packages/worker/src/mcp/run-kody-registry.ts	Adds getEstimatedBytes and assertStorageRunnerWriteWithinEntitlement, applying byte-limit checks before storage_sql/storage_set writes, storage-query capability writes, and package-app storage writes.
Inbound/outbound email storage-bytes enforcement packages/worker/src/email/inbound.ts, packages/worker/src/email/outbound.ts, packages/worker/src/email/inbound-entitlements.workers.test.ts	Gates inbound email receipt on rawSize storage-byte entitlement and outbound sends on an estimated stored-entry size before persisting messages.
Values/secrets/memory/package-registry write checks and userEmail plumbing packages/worker/src/mcp/values/service.ts, packages/worker/src/mcp/secrets/service.ts, packages/worker/src/mcp/memory/service.ts, packages/worker/src/package-registry/service.ts, packages/worker/src/app/handlers/account-secrets.ts, packages/worker/src/mcp/capabilities/{integrations/integration-save,meta/meta-memory-upsert,values/value-set}.ts, packages/worker/src/package-registry/service.node.test.ts	Adds storage-bytes checks before saving values, secrets, memories, and package projections, and passes userEmail/email through save/upsert calls.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

• kentcdodds/kody#619: Extends the same entitlement/quota system that previously left storage_bytes unenforced.
• kentcdodds/kody#627: Also modifies entitlements/service.ts's usage-counting switch/integration points.
• kentcdodds/kody#636: Also modifies the inbound email quota gate in email/inbound.ts.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: adding enforcement for storage-bytes entitlement.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch cursor/storage-bytes-entitlement-enforcement-2791

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🔎 Preview deployed: https://kody-pr-680.kody-a99.workers.dev

Worker: kody-pr-680
D1: kody-pr-680-db
KV: kody-pr-680-oauth-kv

Mocks:

• cloudflare: https://kody-pr-680-mock-cloudflare.kody-a99.workers.dev/__mocks (kody-pr-680-mock-cloudflare)

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 3 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 2ece0c3. Configure here.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

packages/worker/src/entitlements/service.ts (1)

278-514: 🚀 Performance & Scalability | 🔵 Trivial

getCurrent runs ~13 full-table aggregate scans on every guarded write.

readUserD1StorageBytes issues 13 SUM(length(CAST(... AS BLOB))) aggregations across the user's rows, and it is invoked from getCurrent on every planned-user write chokepoint (storage_set/storage_sql, values/secrets/memory upserts, email storage). For users with large email_messages/package_runtime_* histories these blob-length scans are non-trivial and repeat per write.

Consider materializing/caching per-user storage byte usage (e.g., a periodically-refreshed rollup row or short-lived cache) rather than recomputing all surfaces synchronously on each write, given these are denial-of-wallet caps rather than billing-grade accounting.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/worker/src/entitlements/service.ts` around lines 278 - 514,
readUserD1StorageBytes currently does many full-table SUM scans and is called
from getCurrent on every guarded write, causing expensive repeated aggregation
work. Update the storage-usage path so getCurrent does not synchronously
recompute all totals each time; instead, have readUserD1StorageBytes read from a
cached or materialized per-user rollup, refreshed asynchronously or on a short
TTL, and keep the existing callers like getCurrent, storage_set, and storage_sql
pointing to that cheaper lookup.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/worker/src/email/inbound.ts`:
- Around line 324-329: Update the gate-ordering comment in inbound email
handling to match the actual rejection flow in the function that performs the
checks before delivery. The comment should now mention the new storage-bytes
entitlement gate by name, placed between the per-message size cap and the
per-day receive rate, so it stays aligned with the sequence around
assertWithinStorageBytesEntitlement, assertWithinDailyReceiveRateEntitlement,
and assertWithinStoredMessagesEntitlement.

---

Nitpick comments:
In `@packages/worker/src/entitlements/service.ts`:
- Around line 278-514: readUserD1StorageBytes currently does many full-table SUM
scans and is called from getCurrent on every guarded write, causing expensive
repeated aggregation work. Update the storage-usage path so getCurrent does not
synchronously recompute all totals each time; instead, have
readUserD1StorageBytes read from a cached or materialized per-user rollup,
refreshed asynchronously or on a short TTL, and keep the existing callers like
getCurrent, storage_set, and storage_sql pointing to that cheaper lookup.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c2a3c3ce-859c-4010-9ce2-69552b1e5728

📥 Commits

Reviewing files that changed from the base of the PR and between 632585f and 2ece0c3.

📒 Files selected for processing (20)

• docs/contributing/architecture/entitlements.md
• packages/worker/src/app/handlers/account-secrets.ts
• packages/worker/src/email/inbound-entitlements.workers.test.ts
• packages/worker/src/email/inbound.ts
• packages/worker/src/email/outbound.ts
• packages/worker/src/entitlements/entitlements.node.test.ts
• packages/worker/src/entitlements/service.ts
• packages/worker/src/mcp/capabilities/integrations/integration-save.ts
• packages/worker/src/mcp/capabilities/meta/meta-memory-upsert.ts
• packages/worker/src/mcp/capabilities/storage/storage-query.ts
• packages/worker/src/mcp/capabilities/values/value-set.ts
• packages/worker/src/mcp/memory/service.ts
• packages/worker/src/mcp/run-kody-registry.ts
• packages/worker/src/mcp/secrets/service.ts
• packages/worker/src/mcp/values/service.ts
• packages/worker/src/package-registry/service.node.test.ts
• packages/worker/src/package-registry/service.ts
• packages/worker/src/package-runtime/package-app.ts
• packages/worker/src/storage-runner.ts
• packages/worker/src/storage-runner.workers.test.ts