Skip to content

fix(source-granola): 🚨 switch notes cursor to updated_at [BREAKING CHANGE] - #84281

Draft
devin-ai-integration[bot] wants to merge 3 commits into
devin/1786498422-source-granola-boundary-gapfrom
devin/1786498829-source-granola-updated-at-cursor
Draft

fix(source-granola): 🚨 switch notes cursor to updated_at [BREAKING CHANGE]#84281
devin-ai-integration[bot] wants to merge 3 commits into
devin/1786498422-source-granola-boundary-gapfrom
devin/1786498829-source-granola-updated-at-cursor

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

This PR targets PR #84279:

Its diff is scoped to the cursor switch; the base PR contains the non-breaking boundary fix. Retarget this to master once #84279 merges.


What

🚨 Breaking change. source-granola's notes stream switches its incremental cursor from created_at to updated_at.

Granola writes a note's AI summary after the meeting, so a note's content keeps changing after it is created. With a created_at cursor those edits are never re-replicated — the record in the destination is frozen at whatever it looked like the first time it was synced. In the reporter's account, 54 of 142 notes have an updated_at later than their created_at, one drifting 28 days.

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/16931:

This is item I-1 of that issue. Item I-2 (silent record loss at window boundaries) is the non-breaking base PR #84279, which can merge on its own.

A human must run the Connector Breaking Change Release Playbook for this one. It stays a draft until then.

Breaking change details

  • Affected stream: notes (and detailed_notes indirectly, as its parent).
  • Why it is breaking: the cursor field — and therefore the state key — changes. There is no verified migration path (see below), so existing state cannot be carried across.
  • User action required after upgrading: refresh the source schema, clear the notes stream's data, re-sync. Documented in docs/integrations/sources/granola-migrations.md.
  • What happens without that action (verified by test_created_at_state_restarts_from_start_date): the old {"created_at": ...} value is simply ignored because the cursor key is now updated_at, so the stream restarts from start_date and re-emits history. Records are duplicated rather than lost, but the destination is left inconsistent unless the stream is cleared.
  • Version: 0.3.0 — MINOR, since the connector is pre-1.0.
  • upgradeDeadline is 2026-08-26 (~2 weeks out).
  • Verify upgradeDeadline value (currently set to 2026-08-26) — adjust if needed

State migration evaluation

Declarative state_migrations offers LegacyToPerPartitionStateMigration and CustomStateMigration. The former only converts legacy per-partition state and cannot rename a cursor key; the latter needs a custom Python component, which the declarative-first rule rules out here — and the breaking-change gate only counts a migration that tests exercise against the real persisted state shapes. So this ships as a breaking change rather than with a migration.

How

   incremental_sync:
     type: DatetimeBasedCursor
-    cursor_field: created_at
+    cursor_field: updated_at
     start_time_option:
-      field_name: created_after
+      field_name: updated_after
  • updated_at is declared in the notes inline schema (the vendor OpenAPI spec marks it required on NoteSummary, so /v1/notes already returns it; detailed_notes already declared it). Part of sibling issue #16930.
  • No windowing to remove: the API has no updated_before parameter, so a bounded window is not expressible on updated_at. The base PR already moved this cursor to an unbounded shape, which is why it goes first.
  • No separate created_at backfill is needed. updated_at >= created_at always, and it is a required field, so a start_date bound on updated_at still reaches every note a created_at bound would have reached.
  • The user-facing start_date spec field is unchanged and still takes YYYY-MM-DD.
  • integration_tests/sample_state.json moves to the updated_at shape; configured_catalog.json already declared updated_at (it was inconsistent with the manifest until now) and is untouched.

detailed_notes

Left as a SubstreamPartitionRouter child with no incremental_sync, and deliberately without the incremental_dependency: true that the earlier PR #76245 proposed. incremental_dependency only has a useful effect when the child has its own incremental sync; enabling it here would restrict the parent read the child iterates, so details whose parent cursor did not advance in a given sync could be skipped even though Granola had rewritten the note. A full re-read of parents keeps the child complete. AGENTS.md / CONTRIBUTING.md now record that reason instead of the bare deferred_child status, and their incremental table no longer claims the notes cursor is created_at.

Relationship to PR #76245

PR #76245 implemented this same switch but is stale: it was written against 0.1.2 and bumped to 0.2.0, while master has since shipped 0.2.11 with api_budget and concurrency_level, so its metadata, changelog and version numbers no longer apply and it conflicts. I opened a fresh branch instead of reviving it, and reused its migration guide and breakingChanges structure as prior art. Its "human review checklist" asked whether the API accepts full ISO 8601 timestamps and whether an unwindowed sync is safe — the runtime evidence in issue #16931 answers both yes, and that is now encoded in the tests. #76245 can be closed in favor of this pair.

Declarative-First Evaluation

Fully declarative: a DatetimeBasedCursor field change, a RequestOption field name, and an inline schema addition. No components.py, no custom component — including for state migration, as noted above.

Review guide

  1. airbyte-integrations/connectors/source-granola/manifest.yaml — cursor, request option, notes schema
  2. metadata.yaml — 0.3.0 and the breakingChanges entry
  3. docs/integrations/sources/granola-migrations.md — new migration guide
  4. unit_tests/test_notes_incremental_sync.py — cursor behavior tests
  5. AGENTS.md / CONTRIBUTING.md / integration_tests/sample_state.json

Test Coverage

unit_tests/ (added in the base PR) reads the declarative source end to end against a mocked /v1/notes that applies updated_after semantics:

  • test_updated_at_cursor_replicates_later_edits_only — a note edited after the incoming state is re-emitted while an untouched one is not; asserts the exact outgoing query (updated_after=2026-06-01T00:00:00Z, no created_after, no created_before) and that state is emitted under updated_at with the max value.
  • test_created_at_state_restarts_from_start_date — the pre-upgrade state shape is ignored and the sync restarts from start_date (this is what makes the change breaking).
  • test_notes_request_is_unbounded, test_boundary_date_note_is_not_dropped, test_legacy_date_state_is_accepted_and_emits_iso_state carry over from the base PR.

poe test-unit-tests: 5 passed. poe test-integration-tests: 2 passed, 6 skipped (this connector has no acceptance-test-config.yml scenarios — unchanged here).

No live Granola credentials were retrievable in this environment (SECRET_SOURCE-GRANOLA__CREDS), so the API behavior in the mock comes from the reproduction in the issue and the vendor OpenAPI spec, not from a sync I ran. A prerelease validated against a real connection before the breaking release would be worthwhile.

User Impact

Notes edited after creation — which is most of them, since Granola writes the summary after the meeting — are replicated on subsequent incremental syncs instead of staying frozen at their first-synced state. In exchange, existing connections must clear the notes stream and re-sync once when upgrading.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

Rolling back returns the cursor key to created_at, so state written by 0.3.0 would be ignored and the stream would restart from start_date again. A rollback needs the same clear-and-resync treatment as the upgrade.

Link to Devin session: https://app.devin.ai/sessions/ce7406d5bb334fc5a23acbbc19c0db2d

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions

Copy link
Copy Markdown
Contributor

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • 🛠️ Quick Fixes
    • /format-fix - Fixes most formatting issues.
    • /bump-version - Bumps connector versions, scraping changelog description from the PR title.
      • Bump types: patch (default), minor, major, major_rc, rc, promote.
      • The rc type is a smart default: applies minor_rc if stable, or bumps the RC number if already RC.
      • The promote type strips the RC suffix to finalize a release.
      • Example: /bump-version type=rc or /bump-version type=minor
    • /bump-progressive-rollout-version - Alias for /bump-version type=rc. Bumps with an RC suffix and enables progressive rollout.
  • ❇️ AI Testing and Review (internal link: AI-SDLC Docs):
    • /ai-prove-fix - Runs prerelease readiness checks, including testing against customer connections.
    • /ai-canary-prerelease - Rolls out prerelease to 5-10 connections for canary testing.
    • /ai-review - AI-powered PR review for connector safety and quality gates.
  • 📝 AI Documentation:
    • /ai-docs-review - AI-powered documentation review for PRs with connector changes.
    • /ai-create-docs-pr - Creates a documentation PR for connector changes, stacked on the current PR.
  • 🚀 Connector Releases:
    • /publish-connectors-prerelease - Publishes pre-release connector builds (tagged as {version}-preview.{git-sha}) for all modified connectors in the PR.
    • /enable-autopilot-rollouts - Enables autopilot progressive rollouts for the modified connector(s) in the PR, remediating "autopilot rollouts not enabled for {connector-name}" auto-merge blockers. Sets defaultRolloutMode: autopilot and enableProgressiveRollout: true, preserving any existing autopilotConfig.
      • Optional args: connector=<CONNECTOR_NAME> (defaults to the modified connectors in the PR), strategy=fast|slow|default (defaults to fast).
      • Example: /enable-autopilot-rollouts or /enable-autopilot-rollouts connector=source-faker strategy=slow
  • ☕️ JVM connectors:
    • /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
      Example: /update-connector-cdk-version connector=destination-bigquery
  • 🐍 Python connectors:
    • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
    • /poe source example lock - Alias for /poe connector source-example lock.
    • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
    • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.
  • ⚙️ Admin commands:
    • /force-merge reason="<REASON>" - Force merges the PR using admin privileges, bypassing CI checks. Requires a reason.
      Example: /force-merge reason="CI is flaky, tests pass locally"
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@devin-ai-integration devin-ai-integration Bot added the hyd-fix Hydra: ai-fix stage has run label Aug 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Note

Autopilot progressive rollouts are not enabled for the following modified connector(s):

  • source-granola

This is a courtesy heads-up only — it does not block merge or fail any check.
To enable automatic progressive rollouts for the connector(s) above, comment
/enable-autopilot-rollouts on this PR. This sets defaultRolloutMode: autopilot
and enableProgressiveRollout: true in each connector's metadata.yaml,
preserving any existing autopilotConfig.

Co-Authored-By: bot_apk <apk@cognition.ai>
@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Deploy preview for airbyte-docs ready!

Project:airbyte-docs
Status: ✅  Deploy successful!
Preview URL:https://airbyte-docs-30fa2h1r5-airbyte-growth.vercel.app
Latest Commit:15e18a8

Deployed with vercel-action

…n/1786498829-source-granola-updated-at-cursor
@devin-ai-integration devin-ai-integration Bot changed the title 🚨🚨fix(source-granola):🚨🚨 switch notes cursor to updated_at fix(source-granola): switch notes cursor to updated_at Aug 12, 2026
@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

source-granola Connector Test Results

8 tests  +2   6 ✅ +2   6s ⏱️ -1s
2 suites ±0   2 💤 ±0 
2 files   ±0   0 ❌ ±0 

Results for commit 15e18a8. ± Comparison against base commit 123c671.

♻️ This comment has been updated with latest results.

@devin-ai-integration devin-ai-integration Bot changed the title fix(source-granola): switch notes cursor to updated_at fix(source-granola): 🚨 switch notes cursor to updated_at [BREAKING CHANGE] Aug 12, 2026
@airbyte-support-bot

Copy link
Copy Markdown
Contributor

🙋 Escalating this source-granola PR cluster for human consolidation per Hands-Free AI Triage Project triage.

Reason: six open PRs currently target source-granola, including two competing fixes for the same incremental-boundary data loss (#84277 and #84279) plus this breaking cursor change. No pipeline command was triggered on any of them, because advancing them independently would produce duplicated or contradictory changes. A human needs to choose which fix to keep and close the rest as superseded.

Devin session

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🙋 Human Input Needed: agreed that a human should pick the winning boundary fix — that call isn't mine to make. Here's the comparison so it's a quick decision, and what I'd suggest.

The overlap is only on I-2. This PR (the updated_at cursor switch) does not overlap #84277 at all — it is stacked on #84279 purely because both touch the same incremental_sync block. It can be rebased onto whichever I-2 fix survives.

#84279 vs #84277 (same bug, different shape):

  • 84277 keeps the 30-day window and tightens cursor_granularity to PT1S. 84279 removes the windowing entirely (no end_time_option / end_datetime / step).
  • The reason I went unbounded: at PT1S a slice ends at …T23:59:59Z and the next starts at …T00:00:00Z, so records in that final sub-second are still in an unqueried gap — and Granola timestamps carry milliseconds (the reproduction data in airbytehq/airbyte-internal-issues#16931 includes values like …T00:00:00.123Z). It shrinks the gap from 86,400s to 1s rather than closing it. Both PRs independently identified the same cursor_datetime_formats back-compat requirement for legacy bare-date state, so that part is settled either way.
  • The API has no updated_before parameter, so the updated_at cursor has to run unbounded regardless. Landing there now avoids changing the cursor shape twice.
  • Where 84277 is clearly ahead: it adds the missing acceptance-test-config.yml (its R-1 item), which is the reason this bug survived 11 releases. 84279 has nothing equivalent — that work should survive whichever fix wins.

My suggestion: take 84279's unbounded cursor plus 84277's acceptance-test config, and close the loser's cursor change as superseded. I'm happy to do that consolidation into a single PR if someone confirms the direction — I just don't want to close another session's PR or drop its test infrastructure on my own judgement.

Also worth flagging for the same human pass: #76245 is an older, conflicted draft implementing this same updated_at switch against 0.1.2. I opened this PR fresh rather than reviving it and reused its migration guide and breakingChanges structure; it should be closed as superseded.

One more constraint on this PR specifically: it's a breaking release, so it needs an engineer to run the Connector Breaking Change Release Playbook and it stays a draft until then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking-change Don't merge me unless you are ready. connectors/source/granola hyd-fix Hydra: ai-fix stage has run

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants