fix(source-granola): retry throttled and server-error responses with backoff by devin-ai-integration[bot] · Pull Request #84278 · airbytehq/airbyte

devin-ai-integration · 2026-08-12T01:36:39Z

What

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/16928 (certification criterion P-3):

https://github.com/airbytehq/airbyte-internal-issues/issues/16928

#/definitions/base_requester declared only type, url_base and a BearerAuthenticator — there was no error_handler anywhere in the manifest (grep for error_handler|backoff|max_retries on master returns nothing). Retryable responses therefore fell back to the CDK's generic retry behavior with no declared backoff strategy, no max_retries ceiling, and nothing able to read a Retry-After header. Granola publishes a burst limit of 25 requests per 5s and 5 req/s sustained (https://docs.granola.ai/help-center/sharing/integrations/granola-api), so throttling pressure is real.

Retry-After is not confirmed by vendor docs. Granola's OpenAPI spec (https://docs.granola.ai/api-reference/openapi.json) documents only 200/400/401/404/413 responses — no 429 — and contains zero occurrences of Retry-After. No live throttled response was observed, so the header name here is the HTTP-standard one, not one verified against the API. Ordering makes that safe either way (see below), but a follow-up that observes a real 429 should confirm the header name.

How

One DefaultErrorHandler on base_requester (both notes and detailed_notes $ref it, so one handler covers both):

error_handler:
  type: DefaultErrorHandler
  max_retries: 5
  backoff_strategies:
    - type: WaitTimeFromHeader
      header: Retry-After
      max_waiting_time_in_seconds: 60
    - type: ExponentialBackoffStrategy
      factor: 5

Header-first ordering is safe when the header is absent: WaitTimeFromHeader.backoff_time() returns None and DefaultErrorHandler.backoff_time() returns the first non-None result, so a missing header degrades to exponential backoff rather than to a zero wait.

No response_filters — the CDK's DEFAULT_ERROR_MAPPING already retries 429 and 5xx. Sibling sub-issues https://github.com/airbytehq/airbyte-internal-issues/issues/16929 (E-1/E-3 response filters) and https://github.com/airbytehq/airbyte-internal-issues/issues/16925 (413 handling) extend this same handler; they should not add a second one.

Worst-case cumulative backoff (for reconciliation with maxSecondsBetweenMessages, currently unset — tracked in https://github.com/airbytehq/airbyte-internal-issues/issues/16926):

exponential path: 5 + 10 + 20 + 40 + 80 = 155s
header path: capped by max_waiting_time_in_seconds: 60 per attempt → 5 × 60 = 300s

So 300s is the ceiling. Whatever value 16926 sets for maxSecondsBetweenMessages should comfortably exceed 300s; values set by other connectors in this repo range from 2 to 6 hours, so no conflict is expected.

api_budget (25 calls / PT5S) and concurrency_level.default_concurrency: 6 are untouched (P-1/P-2 already pass). The changelog's incorrect claim that 0.2.0-rc.4 reverted concurrency to 5 is left alone — it's tracked in 16926.

Declarative-First Evaluation

Fully declarative: built-in DefaultErrorHandler + WaitTimeFromHeader + ExponentialBackoffStrategy. No components.py exists and none was added.

Test Coverage

No connector-level test. An earlier revision of this PR added a unit_tests/ project asserting the retry/backoff behavior through requests_mock; per reviewer feedback (#84278 (comment)) it was removed, since the behavior it exercised belongs entirely to CDK components (DefaultErrorHandler, WaitTimeFromHeader, ExponentialBackoffStrategy) that carry their own CDK test coverage, and connector tests should cover custom connector behavior instead. This change adds no custom behavior.

For the record, before removal those tests did discriminate: all three cases (429 with Retry-After: 7, 429 without the header, 500) failed against master's manifest — each observed a generic 1s sleep instead of the configured backoff — and passed with the handler in place.

Not run: poe test-integration-tests (needs GSM secrets).

Review guide

airbyte-integrations/connectors/source-granola/manifest.yaml
metadata.yaml (0.2.11 → 0.2.12) and docs/integrations/sources/granola.md

User Impact

Rate-limited and transient server-error responses are retried with backoff instead of failing fast, so syncs are less likely to fail on throttling. No schema, spec, state, or data-scope change — not breaking.

Can this PR be safely reverted and rolled back?

YES 💚
NO ❌

Link to Devin session: https://app.devin.ai/sessions/b16d8d6a9de8433fa68440b15f279a13

Important

⚡ Auto-merge enabled.

This PR is set to merge automatically when all requirements are met.

…backoff Co-Authored-By: bot_apk <apk@cognition.ai>

Co-Authored-By: bot_apk <apk@cognition.ai>

devin-ai-integration · 2026-08-12T01:36:41Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

Disable automatic comment, CI, and merge conflict monitoring

github-actions · 2026-08-12T01:37:06Z

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

🛠️ Quick Fixes
- /format-fix - Fixes most formatting issues.
- /bump-version - Bumps connector versions, scraping changelog description from the PR title.
  - Bump types: patch (default), minor, major, major_rc, rc, promote.
  - The rc type is a smart default: applies minor_rc if stable, or bumps the RC number if already RC.
  - The promote type strips the RC suffix to finalize a release.
  - Example: /bump-version type=rc or /bump-version type=minor
- /bump-progressive-rollout-version - Alias for /bump-version type=rc. Bumps with an RC suffix and enables progressive rollout.
❇️ AI Testing and Review (internal link: AI-SDLC Docs):
- /ai-prove-fix - Runs prerelease readiness checks, including testing against customer connections.
- /ai-canary-prerelease - Rolls out prerelease to 5-10 connections for canary testing.
- /ai-review - AI-powered PR review for connector safety and quality gates.
📝 AI Documentation:
- /ai-docs-review - AI-powered documentation review for PRs with connector changes.
- /ai-create-docs-pr - Creates a documentation PR for connector changes, stacked on the current PR.
🚀 Connector Releases:
- /publish-connectors-prerelease - Publishes pre-release connector builds (tagged as {version}-preview.{git-sha}) for all modified connectors in the PR.
- /enable-autopilot-rollouts - Enables autopilot progressive rollouts for the modified connector(s) in the PR, remediating "autopilot rollouts not enabled for {connector-name}" auto-merge blockers. Sets defaultRolloutMode: autopilot and enableProgressiveRollout: true, preserving any existing autopilotConfig.
  - Optional args: connector=<CONNECTOR_NAME> (defaults to the modified connectors in the PR), strategy=fast|slow|default (defaults to fast).
  - Example: /enable-autopilot-rollouts or /enable-autopilot-rollouts connector=source-faker strategy=slow
☕️ JVM connectors:
- /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
  Example: /update-connector-cdk-version connector=destination-bigquery
🐍 Python connectors:
- /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
- /poe source example lock - Alias for /poe connector source-example lock.
- /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
- /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.
⚙️ Admin commands:
- /force-merge reason="<REASON>" - Force merges the PR using admin privileges, bypassing CI checks. Requires a reason.
  Example: /force-merge reason="CI is flaky, tests pass locally"

📚 Show Repo Guidance

Helpful Resources

Breaking Changes Guide - Breaking changes, migration guides, and upgrade deadlines
Developing Connectors Locally
Managing Connector Secrets
On-Demand Regression Tests
#connector-ci-issues
#connector-publish-updates
#connector-build-statuses

📝 Edit this welcome message.

Co-Authored-By: bot_apk <apk@cognition.ai>

github-actions · 2026-08-12T01:37:27Z

Note

Autopilot progressive rollouts are not enabled for the following modified connector(s):

source-granola

This is a courtesy heads-up only — it does not block merge or fail any check.
To enable automatic progressive rollouts for the connector(s) above, comment
/enable-autopilot-rollouts on this PR. This sets defaultRolloutMode: autopilot
and enableProgressiveRollout: true in each connector's metadata.yaml,
preserving any existing autopilotConfig.

github-actions · 2026-08-12T01:40:12Z

`source-granola` Connector Test Results

3 tests 1 ✅ 3s ⏱️
1 suites 2 💤
1 files 0 ❌

Results for commit e22cc90.

♻️ This comment has been updated with latest results.

github-actions · 2026-08-12T01:41:26Z

Deploy preview for airbyte-docs ready!

Project:	`airbyte-docs`
Status:	✅ Deploy successful!
Preview URL:	https://airbyte-docs-6dzgmqjo3-airbyte-growth.vercel.app
Latest Commit:	`e22cc90`

Deployed with vercel-action

Patrick Nilan (pnilan) · 2026-08-12T01:53:13Z

remove tests from this PR, covered by cdk tests

log hydra feedback: tests should test custom behavior implemented in the code that is being modified, no need to test features that are tested by default in the cdk

👍 On it. Understood — the retry/backoff behavior here is CDK-owned (DefaultErrorHandler, WaitTimeFromHeader, ExponentialBackoffStrategy) and already covered by CDK tests, so a connector-level test that mostly asserts CDK sleep math isn't worth the maintenance. Removing the whole unit_tests/ project from this PR so the diff is just the manifest handler, the version bump, and the changelog row.

Noting the general feedback for the fix workflow: connector-level tests should cover custom behavior in the code being changed, not CDK features that ship with their own tests.

Devin session

☑️ Resolved in e22cc90. The unit_tests/ project is gone; the diff is now just manifest.yaml (the error handler), metadata.yaml (0.2.11 → 0.2.12), and the granola.md changelog row. The PR's Test Coverage section explains why there's no connector-level test.

Devin session

Co-Authored-By: bot_apk <apk@cognition.ai>

devin-ai-integration Bot and others added 2 commits August 12, 2026 01:35

fix(source-granola): retry throttled and server-error responses with …

1f76f52

…backoff Co-Authored-By: bot_apk <apk@cognition.ai>

test(source-granola): parametrize backoff retry test

3116ce4

Co-Authored-By: bot_apk <apk@cognition.ai>

Octavia Squidington III (octavia-squidington-iii) added the connectors/source/granola label Aug 12, 2026

devin-ai-integration Bot assigned Airbyte Support Bot (airbyte-support-bot) Aug 12, 2026

devin-ai-integration Bot added the hyd-fix Hydra: ai-fix stage has run label Aug 12, 2026

docs(source-granola): link changelog entry to PR

a0138a1

Co-Authored-By: bot_apk <apk@cognition.ai>

Patrick Nilan (pnilan) approved these changes Aug 12, 2026

View reviewed changes

Patrick Nilan (pnilan) reviewed Aug 12, 2026

View reviewed changes

Patrick Nilan (pnilan) marked this pull request as ready for review August 12, 2026 01:53

test(source-granola): remove unit tests covered by CDK tests

e22cc90

Co-Authored-By: bot_apk <apk@cognition.ai>

Patrick Nilan (pnilan) enabled auto-merge (squash) August 12, 2026 01:54

Patrick Nilan (pnilan) merged commit fe24599 into master Aug 12, 2026
49 of 52 checks passed

Patrick Nilan (pnilan) deleted the devin/1786498337-source-granola-backoff branch August 12, 2026 01:57

This was referenced Aug 12, 2026

docs(source-granola): update API key model and retry behavior #84283

Merged

test(source-granola): add acceptance-test-config and fix configured catalog #84276

Open

Navigation Menu

Uh oh!

fix(source-granola): retry throttled and server-error responses with backoff - #84278

fix(source-granola): retry throttled and server-error responses with backoff#84278
Patrick Nilan (pnilan) merged 4 commits into
masterfrom
devin/1786498337-source-granola-backoff

devin-ai-integration Bot commented Aug 12, 2026 •

edited by github-actions Bot

Loading

Uh oh!

devin-ai-integration Bot commented Aug 12, 2026

Uh oh!

github-actions Bot commented Aug 12, 2026

PR Slash Commands

Helpful Resources

Uh oh!

github-actions Bot commented Aug 12, 2026

Uh oh!

github-actions Bot commented Aug 12, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 12, 2026 •

edited

Loading

Uh oh!

Patrick Nilan (pnilan) Aug 12, 2026

Uh oh!

devin-ai-integration Bot Aug 12, 2026

Uh oh!

devin-ai-integration Bot Aug 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

devin-ai-integration Bot commented Aug 12, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How

Declarative-First Evaluation

Test Coverage

Review guide

User Impact

Can this PR be safely reverted and rolled back?

Uh oh!

devin-ai-integration Bot commented Aug 12, 2026

🤖 Devin AI Engineer

Uh oh!

github-actions Bot commented Aug 12, 2026

👋 Greetings, Airbyte Team Member!

PR Slash Commands

Helpful Resources

Uh oh!

github-actions Bot commented Aug 12, 2026

Uh oh!

github-actions Bot commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

source-granola Connector Test Results

Uh oh!

github-actions Bot commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Patrick Nilan (pnilan) Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

devin-ai-integration Bot commented Aug 12, 2026 •

edited by github-actions Bot

Loading

github-actions Bot commented Aug 12, 2026 •

edited

Loading

`source-granola` Connector Test Results

github-actions Bot commented Aug 12, 2026 •

edited

Loading