Skip to content

feat(email): add Mailbox parity soak gate - #1126

Merged
kody-bot merged 23 commits into
mainfrom
cursor/mailbox-do-810a
Aug 1, 2026
Merged

kody-bot merged 23 commits into
mainfrom
cursor/mailbox-do-810a

Conversation

@kentcdodds

@kentcdodds kentcdodds commented Aug 1, 2026 •

Copy link
Copy Markdown
Member

Summary

Ships a queue-isolated, resumable Mailbox backfill/parity lane plus a default-off read-cutover gate. D1 remains authoritative; no live reader was switched.

System recap — extends existing primitives (medium risk)

Mode: recap · Base: main @ 0e686a2 · Head: 669e93b

Primitive Impact
mailbox resumable backfill/content replay and mismatch purge/rebuild
scheduled-cron queue-isolated parity every five-minute tick
feature-flags default-off, parity-gated read preparation
d1-app-db progress/soak state and keyset indexes

D1 discovers bounded work; Mailbox is never enumerated. No live read authority changes.

Conductor report

  • STATUS: done
  • What shipped: scalable resumable parity/backfill, negative-drift rebuild, durable soak state, default-off read-adapter preparation
  • Risk self-assessment: medium — bounded scheduled writes and migration state; no authority/read change
  • Merged/deployed: yes — PR #1126, merge, CI, deploy
  • Sibling scope spill: append-only scheduled lane; no index.ts, UserMeter/deletion-fence internals, inbound lifecycle, or retention edits
Open in Web Open in Cursor 

Summary by CodeRabbit

  • New Features

    • Added scheduled mailbox parity checks that backfill messages and delivery events, replay content updates, detect mismatches, and resume interrupted reconciliation.
    • Added safeguards for account deletion, stale checks, count mismatches, and mailbox cleanup.
    • Added a default-off mailbox read cutover feature flag with fail-closed safeguards and success monitoring.
  • Documentation

    • Documented parity processing, reconciliation status, telemetry, migration prerequisites, and read-cutover safeguards.

cursoragent and others added 13 commits August 1, 2026 07:25
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
@coderabbitai

coderabbitai Bot commented Aug 1, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This change adds persisted Mailbox parity state, resumable reconciliation, scheduled processing, mismatch cleanup, telemetry, and a default-off Mailbox read-cutover adapter. It also adds feature-flag registration, migration records, test fixtures, and reconciliation and cutover tests.

Changes

Mailbox parity and reconciliation

Layer / File(s) Summary
Parity state and repository
packages/worker/migrations/0125-mailbox-parity-state.sql, packages/worker/src/email/mailbox-parity-repo.ts, packages/worker/src/users-test-schema.ts
Adds parity columns, indexes, user selection, count queries, keyset pagination, replay cursors, and progress persistence.
Backfill and replay phases
packages/worker/src/email/mailbox-parity-phases.ts
Adds resumable message and delivery-event backfills and bounded content replay with deletion, deadline, and retry handling.
Parity reconciliation workflow
packages/worker/src/email/mailbox-reconcile.ts, packages/worker/src/email/mailbox-reconcile*.node.test.ts, packages/worker/src/test-support/mailbox-reconcile.ts
Adds batch reconciliation, count comparison, mismatch purging, telemetry, checkpoint persistence, sanitized errors, fixtures, and validation.
Scheduled lane integration
packages/worker/src/scheduled/*, packages/worker/src/index.workers.test.ts
Registers and dispatches the mailbox_parity lane with the scheduled timestamp and reconciliation environment.
Read cutover and feature flag
packages/worker/src/email/mailbox-read-cutover.ts, packages/worker/src/feature-flags/*, docs/contributing/architecture/feature-flags.md, packages/worker/src/email/mailbox-read-cutover.node.test.ts
Adds a default-off, fail-closed Mailbox reader gated by parity soak, freshness, identity, and account state.
Architecture and migration records
docs/contributing/architecture/data-storage.md, tools/migration-ledger.json
Documents parity behavior, telemetry, scheduled processing, pending integrations, and migration hashes.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 19.30% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the PR's main change: adding Mailbox parity and soak-gate safeguards.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch cursor/mailbox-do-810a

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

cursoragent and others added 9 commits August 1, 2026 11:51
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
@kody-bot
kody-bot marked this pull request as ready for review August 1, 2026 12:51
@github-actions

github-actions Bot commented Aug 1, 2026 •

Copy link
Copy Markdown
Contributor

🔎 Preview deployed: https://kody-pr-1126.kody-a99.workers.dev

Worker: kody-pr-1126
D1: kody-pr-1126-db
KV: kody-pr-1126-oauth-kv

Mocks:

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
packages/worker/src/email/mailbox-reconcile-mismatch-deletion-error.node.test.ts (1)

167-175: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Align the awaitMailboxMirrorRpc mock with the real return contract.

awaitMailboxMirrorRpc in packages/worker/src/email/mailbox-mirror.ts returns { ok: false; timedOut: true } on timeout. This mock returns { ok: false, result: { status: 'timeout' } }. The production code only reads raced.ok, so the test passes today. A future consumer that reads timedOut would still pass here and fail in production.

Use the real shape.

♻️ Proposed fix
 	mocks.awaitMailboxMirrorRpc.mockImplementation(
 		async (promise: Promise<unknown>) => {
 			try {
 				return { ok: true as const, value: await promise }
 			} catch {
-				return { ok: false as const, result: { status: 'timeout' as const } }
+				return { ok: false as const, timedOut: true as const }
 			}
 		},
 	)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@packages/worker/src/email/mailbox-reconcile-mismatch-deletion-error.node.test.ts`
around lines 167 - 175, Update the awaitMailboxMirrorRpc mock implementation to
return the production timeout shape `{ ok: false, timedOut: true }` in its catch
branch, matching the contract defined by awaitMailboxMirrorRpc while preserving
the successful `{ ok: true, value }` result.
packages/worker/src/email/mailbox-reconcile.ts (1)

588-612: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Remove the duplicate persistCheckpoint calls in the content-replay branches.

state is already assigned from contentPass.state at Line 579, so persistCheckpoint writes the frozen contentReplayUpperAt and contentReplayCursor on every call. The freezeOpenedWindow guard therefore performs an extra UPDATE users that the following call overwrites immediately.

In the budget branch both calls bind identical values, so the first write has no effect at all. In the failed branch the second call only adds lastError. Each affected tick issues two D1 writes instead of one.

Remove the guarded calls. If freezeOpenedWindow is no longer read anywhere, drop it from contentPhaseCheckpoint as well.

♻️ Proposed fix
 		case 'failed':
-			// Durably freeze a newly opened window even when this tick cannot finish it.
-			if (contentCheckpoint.freezeOpenedWindow) {
-				await persistCheckpoint({ matchingSince: null, lastError: null })
-			}
+			// Persisted state already carries the frozen window from contentPass.state.
 			await persistCheckpoint({
 				matchingSince: null,
 				lastError: contentCheckpoint.lastError,
 			})
@@
 		case 'budget':
-			if (contentCheckpoint.freezeOpenedWindow) {
-				await persistCheckpoint({ matchingSince: null, lastError: null })
-			}
 			// Freeze/retain upper + cursor; clear soak on incomplete budget ticks.
 			await persistCheckpoint({ matchingSince: null, lastError: null })
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/worker/src/email/mailbox-reconcile.ts` around lines 588 - 612,
Remove the conditional persistCheckpoint calls guarded by
contentCheckpoint.freezeOpenedWindow in the retryable and budget branches,
leaving one persistCheckpoint call per branch with the required matchingSince
and lastError values. Verify whether freezeOpenedWindow is unused afterward and
remove it from contentPhaseCheckpoint if no longer needed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/worker/src/email/mailbox-reconcile.ts`:
- Around line 35-44: The documentation uses an outdated hourly cadence; update
the descriptions to five-minute cadence. In
packages/worker/src/email/mailbox-reconcile.ts lines 35-44,
packages/worker/src/email/mailbox-parity-repo.ts lines 4-9, and
packages/worker/migrations/0125-mailbox-parity-state.sql line 1, replace each
hourly Mailbox parity/reconcile description with the corresponding five-minute
wording.

In `@packages/worker/src/test-support/mailbox-reconcile.ts`:
- Around line 16-35: Update the module-level mocks for mailbox-live-mirror.ts,
mailbox-parity-events.ts, and mailbox-client.ts to asynchronously import and
spread each module’s original exports, while overriding only the specified mock
symbols. Preserve the existing mailbox-mirror.ts partial-mock pattern and ensure
exports such as mailboxLiveMirrorMaxEvents remain available.

---

Nitpick comments:
In
`@packages/worker/src/email/mailbox-reconcile-mismatch-deletion-error.node.test.ts`:
- Around line 167-175: Update the awaitMailboxMirrorRpc mock implementation to
return the production timeout shape `{ ok: false, timedOut: true }` in its catch
branch, matching the contract defined by awaitMailboxMirrorRpc while preserving
the successful `{ ok: true, value }` result.

In `@packages/worker/src/email/mailbox-reconcile.ts`:
- Around line 588-612: Remove the conditional persistCheckpoint calls guarded by
contentCheckpoint.freezeOpenedWindow in the retryable and budget branches,
leaving one persistCheckpoint call per branch with the required matchingSince
and lastError values. Verify whether freezeOpenedWindow is unused afterward and
remove it from contentPhaseCheckpoint if no longer needed.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 41172aa7-9deb-4c2a-b980-afc3a235978e

📥 Commits

Reviewing files that changed from the base of the PR and between 0e686a2 and 669e93b.

📒 Files selected for processing (24)
  • docs/contributing/architecture/data-storage.md
  • docs/contributing/architecture/feature-flags.md
  • packages/worker/migrations/0125-mailbox-parity-state.sql
  • packages/worker/src/app/handlers/admin-feature-flags.node.test.ts
  • packages/worker/src/app/handlers/session-handler.node.test.ts
  • packages/worker/src/email/mailbox-parity-phases.ts
  • packages/worker/src/email/mailbox-parity-repo.ts
  • packages/worker/src/email/mailbox-read-cutover.node.test.ts
  • packages/worker/src/email/mailbox-read-cutover.ts
  • packages/worker/src/email/mailbox-reconcile-cursor-backfill-content.node.test.ts
  • packages/worker/src/email/mailbox-reconcile-mismatch-deletion-error.node.test.ts
  • packages/worker/src/email/mailbox-reconcile.ts
  • packages/worker/src/email/mailbox-reconcile.workers.test.ts
  • packages/worker/src/feature-flags/registry.node.test.ts
  • packages/worker/src/feature-flags/registry.ts
  • packages/worker/src/feature-flags/service.node.test.ts
  • packages/worker/src/index.workers.test.ts
  • packages/worker/src/mcp/capabilities/access-control.node.test.ts
  • packages/worker/src/mcp/capabilities/admin/admin-feature-flag-capabilities.node.test.ts
  • packages/worker/src/scheduled/scheduled-lanes.node.test.ts
  • packages/worker/src/scheduled/scheduled-lanes.ts
  • packages/worker/src/test-support/mailbox-reconcile.ts
  • packages/worker/src/users-test-schema.ts
  • tools/migration-ledger.json

Comment thread packages/worker/src/email/mailbox-reconcile.ts
Comment thread packages/worker/src/test-support/mailbox-reconcile.ts Outdated
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
@kody-bot
kody-bot merged commit e76b8c9 into main Aug 1, 2026
10 checks passed
@kody-bot
kody-bot deleted the cursor/mailbox-do-810a branch August 1, 2026 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants