Repository navigation
feat(admin): add Mailbox maintenance controls - #1139
Conversation
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Restore deleteEmailMessageById to D1 batch then immediate R2 cleanup with no Mailbox env/waitUntil/mirror. Restore insertEmailMessageWithAttachments signature without mirror forwarding. Drop PR-only delete mirror tests and update data-storage.md: live explicit/retention deletes are repaired by parity purge/rebuild; direct delete wiring remains pending. Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
…doff Add a focused workers test that sends a successful outbound email, waits for the best-effort D1→UserMeter storage shadow, and asserts readStorageBytes matches authoritative users.d1_storage_bytes. Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Exercise handleInboundEmail storage-byte reservation with real UserMeter: capture ExecutionContext waitUntil, drain shadow sync, and assert readStorageBytes matches authoritative users.d1_storage_bytes. Also cover over-quota rejection without meter drift and pre-commit retry without double-reserving D1 storage. Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
📝 WalkthroughWalkthroughMailbox retention now uses a shared owner-bound pass for alarms and immediate execution. An audited admin MCP capability adds status, reconciliation, and bounded retention actions with aggregate metrics, pagination, concurrency limits, and time budgets. ChangesMailbox retention execution
Admin maintenance workflows
Admin MCP integration
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant AdminMCP
participant MaintenanceService
participant D1
participant MailboxDO
AdminMCP->>MaintenanceService: retention request
MaintenanceService->>D1: prune expired rows and find eligible owners
D1-->>MaintenanceService: deletion totals and owner pages
MaintenanceService->>MailboxDO: runRetentionNow(ownerId)
MailboxDO-->>MaintenanceService: mailbox counts and cleanup status
MaintenanceService-->>AdminMCP: aggregate metrics and continuation cursor
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
Co-authored-by: Kent C. Dodds <me+github@kentcdodds.com>
|
🔎 Preview deployed: https://kody-pr-1139.kody-a99.workers.dev Worker: Mocks:
|
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
packages/worker/src/admin/mailbox-maintenance.node.test.ts (1)
16-21: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winReset the hoisted mocks between tests.
mocks.mailboxRpc,mocks.reconcileMailboxParity, and the two prune mocks keep their implementations and call history across tests in this file. Test order then affects results. For example,retention concurrency stays at most 4relies onmocks.mailboxRpcbeing set by itself, but a future test that omits the setup would silently reuse the previous implementation.Add a
beforeEachthat callsvi.resetAllMocks()and re-applies the shared defaults, or enablemockResetin the Vitest config.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/worker/src/admin/mailbox-maintenance.node.test.ts` around lines 16 - 21, Add a beforeEach in the test setup that calls vi.resetAllMocks() and restores the shared default implementations for mocks.mailboxRpc, mocks.reconcileMailboxParity, mocks.pruneUserEmailMessagesForRetention, and mocks.pruneEmailDeliveryEventsForRetention, ensuring each test is isolated.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/worker/src/admin/mailbox-maintenance.ts`:
- Around line 254-296: Bound each launched owner pass in the maintenance worker
around the existing runOwner flow so runOwner cannot keep the returned promise
pending indefinitely. Add a per-owner timeout using input.ownerTimeoutMs when
provided, otherwise the remaining deadline, and resolve timed-out passes with a
failed OwnerRetentionOutcome; clear the timer after either path settles. Keep
the existing concurrency, outcome collection, and deadline-based launch behavior
intact.
- Around line 588-592: Update the surrounding maintenance flow to retain the
hasMore result from pruneUserEmailMessagesForRetention and include it when
computing truncated, alongside stoppedByBudget and pageFull. Ensure
nextStartAfter remains non-null whenever remaining D1 work exists so subsequent
sweeps continue processing expired rows.
---
Nitpick comments:
In `@packages/worker/src/admin/mailbox-maintenance.node.test.ts`:
- Around line 16-21: Add a beforeEach in the test setup that calls
vi.resetAllMocks() and restores the shared default implementations for
mocks.mailboxRpc, mocks.reconcileMailboxParity,
mocks.pruneUserEmailMessagesForRetention, and
mocks.pruneEmailDeliveryEventsForRetention, ensuring each test is isolated.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 83de44cc-8826-4f1c-8d98-a831d950b326
📒 Files selected for processing (10)
docs/contributing/architecture/data-storage.mdpackages/worker/src/admin/mailbox-maintenance.node.test.tspackages/worker/src/admin/mailbox-maintenance.tspackages/worker/src/email/mailbox-do.tspackages/worker/src/email/mailbox-retention.workers.test.tspackages/worker/src/email/mailbox-types.tspackages/worker/src/mcp/capabilities/admin/admin-mailbox-maintenance.node.test.tspackages/worker/src/mcp/capabilities/admin/admin-mailbox-maintenance.tspackages/worker/src/mcp/capabilities/admin/domain.tspackages/worker/src/mcp/capabilities/registry.node.test.ts
| await new Promise<void>((resolve) => { | ||
| const settleIfDone = () => { | ||
| if (inFlight > 0) return | ||
| resolve() | ||
| } | ||
| const launch = () => { | ||
| while ( | ||
| inFlight < input.concurrency && | ||
| nextIndex < input.owners.length && | ||
| input.nowMs() < input.deadlineMs | ||
| ) { | ||
| const owner = input.owners[nextIndex] | ||
| if (!owner) break | ||
| nextIndex += 1 | ||
| lastAttemptedUserId = owner.userId | ||
| inFlight += 1 | ||
| void input | ||
| .runOwner(owner.userId) | ||
| .then((outcome) => { | ||
| outcomes.push(outcome) | ||
| }) | ||
| .catch(() => { | ||
| // runOwner isolates failures; this is a last-resort guard. | ||
| outcomes.push({ status: 'failed' }) | ||
| }) | ||
| .finally(() => { | ||
| inFlight -= 1 | ||
| launch() | ||
| settleIfDone() | ||
| }) | ||
| } | ||
| if ( | ||
| input.nowMs() >= input.deadlineMs && | ||
| nextIndex < input.owners.length | ||
| ) { | ||
| stoppedByBudget = true | ||
| } | ||
| settleIfDone() | ||
| } | ||
| launch() | ||
| }) | ||
|
|
||
| return { outcomes, lastAttemptedUserId, stoppedByBudget } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
The wall-clock budget does not bound in-flight owner passes.
deadlineMs only stops new launches. The returned promise resolves only after every launched runOwner settles. If one Mailbox.runRetentionNow RPC hangs, this function waits indefinitely, and adminMailboxMaintenanceRetentionBudgetMs provides no upper bound on the admin request.
Add a per-owner timeout so each launched pass settles within the budget.
🛡️ Proposed per-owner timeout
async function runOwnersWithBudget(input: {
owners: ReadonlyArray<{ userId: string }>
concurrency: number
deadlineMs: number
nowMs: () => number
runOwner: (userId: string) => Promise<OwnerRetentionOutcome>
+ ownerTimeoutMs?: number
}): Promise<{ void input
.runOwner(owner.userId)
+ .then((outcome) => outcome)
.then((outcome) => {
outcomes.push(outcome)
})Wrap the call so it races a timer, for example:
const withTimeout = async (userId: string) => {
const timeoutMs = input.ownerTimeoutMs ?? Math.max(0, input.deadlineMs - input.nowMs())
let timer: ReturnType<typeof setTimeout> | undefined
try {
return await Promise.race([
input.runOwner(userId),
new Promise<OwnerRetentionOutcome>((resolve) => {
timer = setTimeout(() => resolve({ status: 'failed' as const }), timeoutMs)
}),
])
} finally {
if (timer) clearTimeout(timer)
}
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/worker/src/admin/mailbox-maintenance.ts` around lines 254 - 296,
Bound each launched owner pass in the maintenance worker around the existing
runOwner flow so runOwner cannot keep the returned promise pending indefinitely.
Add a per-owner timeout using input.ownerTimeoutMs when provided, otherwise the
remaining deadline, and resolve timed-out passes with a failed
OwnerRetentionOutcome; clear the timer after either path settles. Keep the
existing concurrency, outcome collection, and deadline-based launch behavior
intact.
| const pageFull = owners.length >= limit | ||
| const truncated = stoppedByBudget || pageFull | ||
| const nextStartAfter = truncated | ||
| ? (lastAttemptedUserId ?? startAfterUserId) | ||
| : null |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
truncated ignores remaining D1 work, so a caller can stop early while expired rows remain.
pruneUserEmailMessagesForRetention returns hasMore (line 497). This function discards it. truncated reflects only the owner page and the budget.
Consider this sequence: the D1 message prune fills its batch and reports hasMore: true; the owner page is shorter than limit; the budget holds. The response then sets truncated: false and nextStartAfter: null. An operator reads that as "sweep complete", but expired D1 rows remain, and every affected owner is still gated by ownerHasPendingExpiredD1Retention, so no Durable Object pass runs for them.
Fold the D1 signal into truncated.
🐛 Proposed fix
const pageFull = owners.length >= limit
- const truncated = stoppedByBudget || pageFull
+ const d1HasMore = d1Messages.hasMore === true
+ const truncated = stoppedByBudget || pageFull || d1HasMore
const nextStartAfter = truncated
? (lastAttemptedUserId ?? startAfterUserId)
: null📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const pageFull = owners.length >= limit | |
| const truncated = stoppedByBudget || pageFull | |
| const nextStartAfter = truncated | |
| ? (lastAttemptedUserId ?? startAfterUserId) | |
| : null | |
| const pageFull = owners.length >= limit | |
| const d1HasMore = d1Messages.hasMore === true | |
| const truncated = stoppedByBudget || pageFull || d1HasMore | |
| const nextStartAfter = truncated | |
| ? (lastAttemptedUserId ?? startAfterUserId) | |
| : null |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/worker/src/admin/mailbox-maintenance.ts` around lines 588 - 592,
Update the surrounding maintenance flow to retain the hasMore result from
pruneUserEmailMessagesForRetention and include it when computing truncated,
alongside stoppedByBudget and pageFull. Ensure nextStartAfter remains non-null
whenever remaining D1 work exists so subsequent sweeps continue processing
expired rows.
Summary
Ships audited Mailbox status/reconcile and owner-gated D1-before-DO natural retention controls.
Production evidence
Forced one complete retention page across all 3 tracked owners: D1 and Mailbox both pruned zero naturally eligible rows; Mailbox counts stayed 100 threads / 185 messages / 48 attachments / 269 events; no blob failures, expired remaining, pending D1 owners, or owner failures. Production currently has no messages older than 365 days and no delivery events older than 90 days, so actual age-qualified blob-before-row deletion remains elapsed-time/test coverage.
Conductor report
Summary by CodeRabbit
New Features
Bug Fixes