google: handle expired history and log skipped items#1174
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughAdds a resilient history-fetching function to handle Gmail history API expiry (404 errors) and large sync gaps, with backoff logic and improved logging. Includes comprehensive test coverage for error scenarios and large history gaps. Maintains existing API surface while enhancing internal robustness. Changes
Sequence Diagram(s)sequenceDiagram
participant WebhookHandler as Webhook Handler
participant FetchResilient as fetchGmailHistoryResilient
participant GmailClient as Gmail Client
participant Logger as Logger
participant DB as Prisma (Database)
WebhookHandler->>FetchResilient: Call with lastSyncedHistoryId, historyId
FetchResilient->>FetchResilient: Compute safe startHistoryId<br/>(with 500-item backoff)
alt Large Gap Detected
FetchResilient->>Logger: warn(lastSyncedHistoryId, gap size)
end
FetchResilient->>GmailClient: getHistory(startHistoryId)
alt 404 / History Expired
GmailClient-->>FetchResilient: Error (404)
FetchResilient->>Logger: warn(history expired)
FetchResilient-->>WebhookHandler: {status: "expired"}
WebhookHandler->>DB: Update lastSyncedHistoryId → historyId
WebhookHandler-->>WebhookHandler: Return {ok: true}
else Success
GmailClient-->>FetchResilient: history data
FetchResilient-->>WebhookHandler: {status: "ok", history, startHistoryId}
WebhookHandler->>WebhookHandler: Process history messages
WebhookHandler->>DB: Update lastSyncedHistoryId
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Handle expired Gmail history IDs and log skipped gaps >500 in
|
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
| err.status ?? | ||
| err.code; | ||
|
|
||
| return statusCode === 404; |
There was a problem hiding this comment.
Suggestion: Normalize numeric inputs from external sources before use—coerce to numbers, check for NaN, and apply safe defaults—to prevent missed expiry detection and NaN propagation.
| return statusCode === 404; | |
| return statusCode === 404 || statusCode === "404"; |
🚀 Want me to fix this? Reply ex: "fix it for me".
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
![baz-reviewer[bot]](https://avatars.githubusercontent.com/in/933528?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
apps/web/app/api/google/webhook/process-history.test.ts (3)
30-36: Consider using the standard Prisma mock.Per the coding guidelines, Prisma should be mocked using the provided mock from
@/utils/__mocks__/prisma. While the current inline mock works, using the standard mock provides consistency across the test suite.🔎 Proposed refactor
-vi.mock("@/utils/prisma", () => ({ - default: { - emailAccount: { - update: vi.fn().mockResolvedValue({}), - }, - }, -})); +vi.mock("@/utils/prisma");Then import and configure the mock in your test:
import prisma from "@/utils/prisma"; // In beforeEach or test setup: vi.mocked(prisma.emailAccount.update).mockResolvedValue({} as any);
50-54: Use test helper functions for creating mock data.Per the coding guidelines and learnings, test data should be created using helper functions from
@/__tests__/helpers(such asgetEmailAccount) instead of inline object literals. This improves consistency and maintainability across the test suite.Based on learnings, use existing helpers from
@/__tests__/helpers.tsfor consistency.🔎 Proposed refactor
+import { getEmailAccount } from "@/__tests__/helpers"; + it("should reset lastSyncedHistoryId when Gmail returns 404 (expired historyId)", async () => { const email = "user@test.com"; const historyId = 2000; - const emailAccount = { - id: "account-123", - email, - lastSyncedHistoryId: "1000", - }; + const emailAccount = getEmailAccount({ + id: "account-123", + email, + lastSyncedHistoryId: "1000", + });
98-102: Use test helper functions for creating mock data.Same as the previous test case: use helper functions from
@/__tests__/helperslikegetEmailAccountinstead of inline object literals for consistency.Based on learnings, use existing helpers from
@/__tests__/helpers.ts.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
🧰 Additional context used
📓 Path-based instructions (20)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/data-fetching.mdc)
**/*.{ts,tsx}: For API GET requests to server, use theswrpackage
Useresult?.serverErrorwithtoastErrorfrom@/components/Toastfor error handling in async operations
**/*.{ts,tsx}: Use wrapper functions for Gmail message operations (get, list, batch, etc.) from @/utils/gmail/message.ts instead of direct API calls
Use wrapper functions for Gmail thread operations from @/utils/gmail/thread.ts instead of direct API calls
Use wrapper functions for Gmail label operations from @/utils/gmail/label.ts instead of direct API calls
**/*.{ts,tsx}: For early access feature flags, create hooks using the naming conventionuse[FeatureName]Enabledthat return a boolean fromuseFeatureFlagEnabled("flag-key")
For A/B test variant flags, create hooks using the naming conventionuse[FeatureName]Variantthat define variant types, useuseFeatureFlagVariantKey()with type casting, and provide a default "control" fallback
Use kebab-case for PostHog feature flag keys (e.g.,inbox-cleaner,pricing-options-2)
Always define types for A/B test variant flags (e.g.,type PricingVariant = "control" | "variant-a" | "variant-b") and provide type safety through type casting
**/*.{ts,tsx}: Don't use primitive type aliases or misleading types
Don't use empty type parameters in type aliases and interfaces
Don't use this and super in static contexts
Don't use any or unknown as type constraints
Don't use the TypeScript directive @ts-ignore
Don't use TypeScript enums
Don't export imported variables
Don't add type annotations to variables, parameters, and class properties that are initialized with literal expressions
Don't use TypeScript namespaces
Don't use non-null assertions with the!postfix operator
Don't use parameter properties in class constructors
Don't use user-defined types
Useas constinstead of literal types and type annotations
Use eitherT[]orArray<T>consistently
Initialize each enum member value explicitly
Useexport typefor types
Use `impo...
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.cursor/rules/prisma-enum-imports.mdc)
Always import Prisma enums from
@/generated/prisma/enumsinstead of@/generated/prisma/clientto avoid Next.js bundling errors in client componentsImport Prisma using the project's centralized utility:
import prisma from '@/utils/prisma'
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
apps/web/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/project-structure.mdc)
Import specific lodash functions rather than entire lodash library to minimize bundle size (e.g.,
import groupBy from 'lodash/groupBy')
apps/web/**/*.{ts,tsx}: Use TypeScript with strict null checks
Do not export types/interfaces that are only used within the same file. Export later if needed
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
apps/web/app/api/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/security-audit.mdc)
apps/web/app/api/**/*.{ts,tsx}: API routes must usewithAuth,withEmailAccount, orwithErrormiddleware for authentication
All database queries must include user scoping withemailAccountIdoruserIdfiltering in WHERE clauses
Request parameters must be validated before use; avoid direct parameter usage without type checking
Use generic error messages instead of revealing internal details; throwSafeErrorinstead of exposing user IDs, resource IDs, or system information
API routes should only return necessary fields usingselectin database queries to prevent unintended information disclosure
Cron endpoints must usehasCronSecretorhasPostCronSecretto validate cron requests and prevent unauthorized access
Request bodies should use Zod schemas for validation to ensure type safety and prevent injection attacks
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
**/app/api/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/security.mdc)
**/app/api/**/*.ts: ALL API routes that handle user data MUST use appropriate middleware: usewithEmailAccountfor email-scoped operations, usewithAuthfor user-scoped operations, or usewithErrorwith proper validation for public/custom auth endpoints
UsewithEmailAccountmiddleware for operations scoped to a specific email account, including reading/writing emails, rules, schedules, or any operation usingemailAccountId
UsewithAuthmiddleware for user-level operations such as user settings, API keys, and referrals that use onlyuserId
UsewithErrormiddleware only for public endpoints, custom authentication logic, or cron endpoints. For cron endpoints, MUST usehasCronSecret()orhasPostCronSecret()validation
Cron endpoints without proper authentication can be triggered by anyone. CRITICAL: All cron endpoints MUST validate cron secret usinghasCronSecret(request)orhasPostCronSecret(request)and capture unauthorized attempts withcaptureException()
Always validate request bodies using Zod schemas to ensure type safety and prevent invalid data from reaching database operations
Maintain consistent error response format across all API routes to avoid information disclosure while providing meaningful error feedback
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/security.mdc)
**/*.ts: ALL database queries MUST be scoped to the authenticated user/account by including user/account filtering in WHERE clauses to prevent unauthorized data access
Always validate that resources belong to the authenticated user before performing operations, using ownership checks in WHERE clauses or relationships
Always validate all input parameters for type, format, and length before using them in database queries
Use SafeError for error responses to prevent information disclosure. Generic error messages should not reveal internal IDs, logic, or resource ownership details
Only return necessary fields in API responses using Prisma'sselectoption. Never expose sensitive data such as password hashes, private keys, or system flags
Prevent Insecure Direct Object References (IDOR) by validating resource ownership before operations. AllfindUnique/findFirstcalls MUST include ownership filters
Prevent mass assignment vulnerabilities by explicitly whitelisting allowed fields in update operations instead of accepting all user-provided data
Prevent privilege escalation by never allowing users to modify system fields, ownership fields, or admin-only attributes through user input
AllfindManyqueries MUST be scoped to the user's data by including appropriate WHERE filters to prevent returning data from other users
Use Prisma relationships for access control by leveraging nested where clauses (e.g.,emailAccount: { id: emailAccountId }) to validate ownership
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
**/*.{tsx,ts}
📄 CodeRabbit inference engine (.cursor/rules/ui-components.mdc)
**/*.{tsx,ts}: Use Shadcn UI and Tailwind for components and styling
Usenext/imagepackage for images
For API GET requests to server, use theswrpackage with hooks likeuseSWRto fetch data
For text inputs, use theInputcomponent withregisterPropsfor form integration and error handling
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
**/*.{tsx,ts,css}
📄 CodeRabbit inference engine (.cursor/rules/ui-components.mdc)
Implement responsive design with Tailwind CSS using a mobile-first approach
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
**/*.{js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/ultracite.mdc)
**/*.{js,jsx,ts,tsx}: Don't useaccessKeyattribute on any HTML element
Don't setaria-hidden="true"on focusable elements
Don't add ARIA roles, states, and properties to elements that don't support them
Don't use distracting elements like<marquee>or<blink>
Only use thescopeprop on<th>elements
Don't assign non-interactive ARIA roles to interactive HTML elements
Make sure label elements have text content and are associated with an input
Don't assign interactive ARIA roles to non-interactive HTML elements
Don't assigntabIndexto non-interactive HTML elements
Don't use positive integers fortabIndexproperty
Don't include "image", "picture", or "photo" in img alt prop
Don't use explicit role property that's the same as the implicit/default role
Make static elements with click handlers use a valid role attribute
Always include atitleelement for SVG elements
Give all elements requiring alt text meaningful information for screen readers
Make sure anchors have content that's accessible to screen readers
AssigntabIndexto non-interactive HTML elements witharia-activedescendant
Include all required ARIA attributes for elements with ARIA roles
Make sure ARIA properties are valid for the element's supported roles
Always include atypeattribute for button elements
Make elements with interactive roles and handlers focusable
Give heading elements content that's accessible to screen readers (not hidden witharia-hidden)
Always include alangattribute on the html element
Always include atitleattribute for iframe elements
AccompanyonClickwith at least one of:onKeyUp,onKeyDown, oronKeyPress
AccompanyonMouseOver/onMouseOutwithonFocus/onBlur
Include caption tracks for audio and video elements
Use semantic elements instead of role attributes in JSX
Make sure all anchors are valid and navigable
Ensure all ARIA properties (aria-*) are valid
Use valid, non-abstract ARIA roles for elements with ARIA roles
Use valid AR...
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
**/*.{test,spec}.{js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/ultracite.mdc)
**/*.{test,spec}.{js,jsx,ts,tsx}: Don't nest describe() blocks too deeply in test files
Don't use callbacks in asynchronous tests and hooks
Don't have duplicate hooks in describe blocks
Don't use export or module.exports in test files
Don't use focused tests
Make sure the assertion function, like expect, is placed inside an it() function call
Don't use disabled tests
Files:
apps/web/app/api/google/webhook/process-history.test.ts
!(pages/_document).{jsx,tsx}
📄 CodeRabbit inference engine (.cursor/rules/ultracite.mdc)
Don't use the next/head module in pages/_document.js on Next.js projects
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
**/*.{js,ts,jsx,tsx}
📄 CodeRabbit inference engine (.cursor/rules/utilities.mdc)
**/*.{js,ts,jsx,tsx}: Use lodash utilities for common operations (arrays, objects, strings)
Import specific lodash functions to minimize bundle size (e.g.,import groupBy from 'lodash/groupBy')
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
apps/web/**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (apps/web/CLAUDE.md)
apps/web/**/*.{ts,tsx,js,jsx}: Use@/path aliases for imports from project root
Prefer self-documenting code over comments; use descriptive variable and function names instead of explaining intent with comments
Add helper functions to the bottom of files, not the top
All imports go at the top of files, no mid-file dynamic imports
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
apps/web/app/**/*.{ts,tsx}
📄 CodeRabbit inference engine (apps/web/CLAUDE.md)
Follow NextJS app router structure with (app) directory
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
apps/web/**/*.{ts,tsx,js,jsx,json,css}
📄 CodeRabbit inference engine (apps/web/CLAUDE.md)
Format code with Prettier
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
apps/web/**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (apps/web/CLAUDE.md)
Co-locate test files next to source files (e.g.,
utils/example.test.ts). Only E2E and AI tests go in__tests__/
Files:
apps/web/app/api/google/webhook/process-history.test.ts
apps/web/**/*.{example,ts,json}
📄 CodeRabbit inference engine (apps/web/CLAUDE.md)
Add environment variables to
.env.example,env.ts, andturbo.json
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
apps/web/app/api/**/*.ts
📄 CodeRabbit inference engine (apps/web/CLAUDE.md)
apps/web/app/api/**/*.ts: Create GET API routes wrapped withwithAuthorwithEmailAccountmiddleware for fetching data
Export response types from GET API routes usingexport type GetXResponse = Awaited<ReturnType<typeof getData>>
Files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
**/*.test.{js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/notes.mdc)
Co-locate test files next to source files (e.g.,
utils/example.test.ts). Only E2E and AI tests go in__tests__/
Files:
apps/web/app/api/google/webhook/process-history.test.ts
**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/testing.mdc)
**/*.test.{ts,tsx}: Usevitestas the testing framework
Colocate test files next to the tested file with.test.tsor.test.tsxnaming convention (e.g.,dir/format.tsanddir/format.test.ts)
Mockserver-onlyusingvi.mock("server-only", () => ({}))
Mock Prisma usingvi.mock("@/utils/prisma")and the provided mock from@/utils/__mocks__/prisma
Use test helper functionsgetEmail,getEmailAccount, andgetRulefrom@/__tests__/helpersfor creating mock data
Clear all mocks between tests usingbeforeEach(() => { vi.clearAllMocks(); })
Use descriptive test names that clearly indicate what is being tested
Do not mock the Logger in tests
Files:
apps/web/app/api/google/webhook/process-history.test.ts
🧠 Learnings (8)
📚 Learning: 2026-01-01T10:42:29.767Z
Learnt from: CR
Repo: elie222/inbox-zero PR: 0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2026-01-01T10:42:29.767Z
Learning: Applies to **/*.test.{ts,tsx} : Use test helper functions `getEmail`, `getEmailAccount`, and `getRule` from `@/__tests__/helpers` for creating mock data
Applied to files:
apps/web/app/api/google/webhook/process-history.test.ts
📚 Learning: 2025-11-25T14:37:56.430Z
Learnt from: CR
Repo: elie222/inbox-zero PR: 0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-11-25T14:37:56.430Z
Learning: Applies to apps/web/__tests__/**/*.test.ts : Prefer using existing helpers from `@/__tests__/helpers.ts` (`getEmailAccount`, `getEmail`, `getRule`, `getMockMessage`, `getMockExecutedRule`) instead of creating custom test data helpers
Applied to files:
apps/web/app/api/google/webhook/process-history.test.ts
📚 Learning: 2025-11-25T14:37:22.660Z
Learnt from: CR
Repo: elie222/inbox-zero PR: 0
File: .cursor/rules/gmail-api.mdc:0-0
Timestamp: 2025-11-25T14:37:22.660Z
Learning: Applies to apps/web/utils/gmail/**/*.{ts,tsx} : Keep Gmail provider-specific implementation details isolated within the apps/web/utils/gmail/ directory
Applied to files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
📚 Learning: 2026-01-01T10:42:29.767Z
Learnt from: CR
Repo: elie222/inbox-zero PR: 0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2026-01-01T10:42:29.767Z
Learning: Applies to **/*.test.{ts,tsx} : Mock Prisma using `vi.mock("@/utils/prisma")` and the provided mock from `@/utils/__mocks__/prisma`
Applied to files:
apps/web/app/api/google/webhook/process-history.test.ts
📚 Learning: 2025-11-25T14:37:22.660Z
Learnt from: CR
Repo: elie222/inbox-zero PR: 0
File: .cursor/rules/gmail-api.mdc:0-0
Timestamp: 2025-11-25T14:37:22.660Z
Learning: Applies to **/*.{ts,tsx} : Use wrapper functions for Gmail thread operations from @/utils/gmail/thread.ts instead of direct API calls
Applied to files:
apps/web/app/api/google/webhook/process-history.test.tsapps/web/app/api/google/webhook/process-history.ts
📚 Learning: 2025-11-25T14:37:22.660Z
Learnt from: CR
Repo: elie222/inbox-zero PR: 0
File: .cursor/rules/gmail-api.mdc:0-0
Timestamp: 2025-11-25T14:37:22.660Z
Learning: Applies to **/*.{ts,tsx} : Use wrapper functions for Gmail message operations (get, list, batch, etc.) from @/utils/gmail/message.ts instead of direct API calls
Applied to files:
apps/web/app/api/google/webhook/process-history.ts
📚 Learning: 2025-11-25T14:37:22.660Z
Learnt from: CR
Repo: elie222/inbox-zero PR: 0
File: .cursor/rules/gmail-api.mdc:0-0
Timestamp: 2025-11-25T14:37:22.660Z
Learning: Applies to apps/web/utils/gmail/**/*.{ts,tsx} : Always use wrapper functions from @/utils/gmail/ for Gmail API operations instead of direct provider API calls
Applied to files:
apps/web/app/api/google/webhook/process-history.ts
📚 Learning: 2025-11-25T14:37:22.660Z
Learnt from: CR
Repo: elie222/inbox-zero PR: 0
File: .cursor/rules/gmail-api.mdc:0-0
Timestamp: 2025-11-25T14:37:22.660Z
Learning: Applies to **/*.{ts,tsx} : Use wrapper functions for Gmail label operations from @/utils/gmail/label.ts instead of direct API calls
Applied to files:
apps/web/app/api/google/webhook/process-history.ts
🧬 Code graph analysis (1)
apps/web/app/api/google/webhook/process-history.ts (3)
apps/web/utils/webhook/validate-webhook-account.ts (1)
ValidatedWebhookAccountData(104-106)apps/web/utils/logger.ts (1)
Logger(6-6)apps/web/utils/gmail/history.ts (1)
getHistory(4-22)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: Baz Reviewer
- GitHub Check: cubic · AI code reviewer
- GitHub Check: Macroscope - Correctness Check
- GitHub Check: test
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (7)
apps/web/app/api/google/webhook/process-history.test.ts (2)
79-92: LGTM! Test correctly validates 404 handling.The test properly verifies that when Gmail returns a 404 (expired history ID), the system:
- Updates
lastSyncedHistoryIdto the currenthistoryIdto recover from the expiry- Returns a successful response
This aligns with the PR objectives for handling expired history IDs.
126-135: LGTM! Test correctly validates large gap logging.The test properly verifies the skip-path logging when the history gap exceeds 500 items. The calculation is correct:
- Gap: 2000 - 1000 = 1000 items
- Effective start: max(1000, 2000 - 500) = 1500
- Skipped: 1500 - 1000 = 500 items
This validates the 500-item cap mechanism described in the PR objectives.
apps/web/app/api/google/webhook/process-history.ts (5)
16-16: LGTM! Type imports are appropriate.The new type imports (
ValidatedWebhookAccountDataandgmail_v1) are properly used in the newfetchGmailHistoryResilientfunction for type safety.Also applies to: 20-20
81-95: LGTM! Expired history handling is robust.The code correctly handles expired history IDs by:
- Detecting the expired status from the resilient fetch
- Updating
lastSyncedHistoryIdto the current webhookhistoryIdto reset the sync point- Returning success to prevent retry loops
This implements the 404 recovery mechanism described in the PR objectives.
100-102: LGTM! Improved logging for observability.The updated logging now includes
startHistoryIdfrom the resilient fetch result, providing better visibility into the actual history range being processed after applying the backoff logic.Also applies to: 122-124
257-267: LGTM! Robust error detection logic.The helper function correctly checks multiple possible locations for the 404 status code, accounting for different error formats that might be returned by the Gmail API or HTTP client.
269-342: LGTM! Well-designed resilient history fetching.The
fetchGmailHistoryResilientfunction implements the key improvements described in the PR objectives:
- 500-item backoff cap: Prevents runaway processing by limiting lookback to
webhookHistoryId - 500when the gap is large- Expired history handling: Catches 404 errors and returns
"expired"status for graceful recovery- Enhanced logging: Warns when items are skipped and logs the gap size for observability
The logic is correct:
- For a gap of 1000 (e.g., lastSynced=1000, webhook=2000): starts at max(1000, 1500) = 1500, skipping 500 items
- For a gap of 200: starts at max(1000, 1800) = 1800, processing normally with no skip
1 participant
User description
google: handle expired history and log skipped items
Add logging when history items are skipped due to the 500-item cap and handle 404 errors gracefully when historyId expires.
fetchGmailHistoryResilienthelperlastSyncedHistoryIdto current pointGenerated description
Below is a concise technical summary of the changes proposed in this PR:
graph LR processHistoryForUser_("processHistoryForUser"):::modified fetchGmailHistoryResilient_("fetchGmailHistoryResilient"):::added processHistory_("processHistory"):::modified updateLastSyncedHistoryId_("updateLastSyncedHistoryId"):::modified getHistory_("getHistory"):::modified isHistoryIdExpiredError_("isHistoryIdExpiredError"):::added GMAIL_("GMAIL"):::modified processHistoryForUser_ -- "Now retrieves resilient historyResult handling expired IDs." --> fetchGmailHistoryResilient_ processHistoryForUser_ -- "Now passes resiliently fetched history to processHistory." --> processHistory_ processHistoryForUser_ -- "Persist current webhookHistoryId when history expired." --> updateLastSyncedHistoryId_ fetchGmailHistoryResilient_ -- "Calls getHistory with computed startHistoryId and filters." --> getHistory_ fetchGmailHistoryResilient_ -- "Detects 404 expired-history errors to reset sync." --> isHistoryIdExpiredError_ fetchGmailHistoryResilient_ -- "Uses Gmail API to list up to 500 history entries." --> GMAIL_ getHistory_ -- "Queries Gmail history:list endpoint for provided startHistoryId." --> GMAIL_ classDef added stroke:#15AA7A classDef removed stroke:#CD5270 classDef modified stroke:#EDAC4C linkStyle default stroke:#CBD5E1,font-size:13pxRefactor the Gmail history fetching logic into a new
fetchGmailHistoryResilienthelper function, enhancing theprocessHistoryForUserfunction to gracefully handle expired Gmail history IDs by resetting the sync point and to log warnings when history items are skipped due to large gaps.Latest Contributors(2)
Summary by CodeRabbit
Bug Fixes
Tests
✏️ Tip: You can customize this high-level summary in your review settings.