fix(source-granola): 🚨 switch notes cursor to updated_at [BREAKING CHANGE] - #84281
Conversation
Co-Authored-By: bot_apk <apk@cognition.ai>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksPR Slash CommandsAirbyte Maintainers (that's you!) can execute the following slash commands on your PR:
📚 Show Repo GuidanceHelpful Resources
|
|
Note Autopilot progressive rollouts are not enabled for the following modified connector(s):
This is a courtesy heads-up only — it does not block merge or fail any check. |
Co-Authored-By: bot_apk <apk@cognition.ai>
|
Deploy preview for airbyte-docs ready!
Deployed with vercel-action |
…n/1786498829-source-granola-updated-at-cursor
|
🙋 Escalating this Reason: six open PRs currently target |
|
🙋 Human Input Needed: agreed that a human should pick the winning boundary fix — that call isn't mine to make. Here's the comparison so it's a quick decision, and what I'd suggest. The overlap is only on I-2. This PR (the #84279 vs #84277 (same bug, different shape):
My suggestion: take 84279's unbounded cursor plus 84277's acceptance-test config, and close the loser's cursor change as superseded. I'm happy to do that consolidation into a single PR if someone confirms the direction — I just don't want to close another session's PR or drop its test infrastructure on my own judgement. Also worth flagging for the same human pass: #76245 is an older, conflicted draft implementing this same One more constraint on this PR specifically: it's a breaking release, so it needs an engineer to run the Connector Breaking Change Release Playbook and it stays a draft until then. |
This PR targets PR #84279:
Its diff is scoped to the cursor switch; the base PR contains the non-breaking boundary fix. Retarget this to
masteronce #84279 merges.What
🚨 Breaking change.
source-granola'snotesstream switches its incremental cursor fromcreated_attoupdated_at.Granola writes a note's AI summary after the meeting, so a note's content keeps changing after it is created. With a
created_atcursor those edits are never re-replicated — the record in the destination is frozen at whatever it looked like the first time it was synced. In the reporter's account, 54 of 142 notes have anupdated_atlater than theircreated_at, one drifting 28 days.Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/16931:
This is item I-1 of that issue. Item I-2 (silent record loss at window boundaries) is the non-breaking base PR #84279, which can merge on its own.
A human must run the Connector Breaking Change Release Playbook for this one. It stays a draft until then.
Breaking change details
notes(anddetailed_notesindirectly, as its parent).notesstream's data, re-sync. Documented indocs/integrations/sources/granola-migrations.md.test_created_at_state_restarts_from_start_date): the old{"created_at": ...}value is simply ignored because the cursor key is nowupdated_at, so the stream restarts fromstart_dateand re-emits history. Records are duplicated rather than lost, but the destination is left inconsistent unless the stream is cleared.upgradeDeadlineis2026-08-26(~2 weeks out).State migration evaluation
Declarative
state_migrationsoffersLegacyToPerPartitionStateMigrationandCustomStateMigration. The former only converts legacy per-partition state and cannot rename a cursor key; the latter needs a custom Python component, which the declarative-first rule rules out here — and the breaking-change gate only counts a migration that tests exercise against the real persisted state shapes. So this ships as a breaking change rather than with a migration.How
incremental_sync: type: DatetimeBasedCursor - cursor_field: created_at + cursor_field: updated_at start_time_option: - field_name: created_after + field_name: updated_afterupdated_atis declared in thenotesinline schema (the vendor OpenAPI spec marks it required onNoteSummary, so/v1/notesalready returns it;detailed_notesalready declared it). Part of sibling issue #16930.updated_beforeparameter, so a bounded window is not expressible onupdated_at. The base PR already moved this cursor to an unbounded shape, which is why it goes first.created_atbackfill is needed.updated_at >= created_atalways, and it is a required field, so astart_datebound onupdated_atstill reaches every note acreated_atbound would have reached.start_datespec field is unchanged and still takesYYYY-MM-DD.integration_tests/sample_state.jsonmoves to theupdated_atshape;configured_catalog.jsonalready declaredupdated_at(it was inconsistent with the manifest until now) and is untouched.detailed_notesLeft as a
SubstreamPartitionRouterchild with noincremental_sync, and deliberately without theincremental_dependency: truethat the earlier PR #76245 proposed.incremental_dependencyonly has a useful effect when the child has its own incremental sync; enabling it here would restrict the parent read the child iterates, so details whose parent cursor did not advance in a given sync could be skipped even though Granola had rewritten the note. A full re-read of parents keeps the child complete.AGENTS.md/CONTRIBUTING.mdnow record that reason instead of the baredeferred_childstatus, and their incremental table no longer claims thenotescursor iscreated_at.Relationship to PR #76245
PR #76245 implemented this same switch but is stale: it was written against 0.1.2 and bumped to 0.2.0, while master has since shipped 0.2.11 with
api_budgetandconcurrency_level, so its metadata, changelog and version numbers no longer apply and it conflicts. I opened a fresh branch instead of reviving it, and reused its migration guide andbreakingChangesstructure as prior art. Its "human review checklist" asked whether the API accepts full ISO 8601 timestamps and whether an unwindowed sync is safe — the runtime evidence in issue #16931 answers both yes, and that is now encoded in the tests. #76245 can be closed in favor of this pair.Declarative-First Evaluation
Fully declarative: a
DatetimeBasedCursorfield change, aRequestOptionfield name, and an inline schema addition. Nocomponents.py, no custom component — including for state migration, as noted above.Review guide
airbyte-integrations/connectors/source-granola/manifest.yaml— cursor, request option,notesschemametadata.yaml— 0.3.0 and thebreakingChangesentrydocs/integrations/sources/granola-migrations.md— new migration guideunit_tests/test_notes_incremental_sync.py— cursor behavior testsAGENTS.md/CONTRIBUTING.md/integration_tests/sample_state.jsonTest Coverage
unit_tests/(added in the base PR) reads the declarative source end to end against a mocked/v1/notesthat appliesupdated_aftersemantics:test_updated_at_cursor_replicates_later_edits_only— a note edited after the incoming state is re-emitted while an untouched one is not; asserts the exact outgoing query (updated_after=2026-06-01T00:00:00Z, nocreated_after, nocreated_before) and that state is emitted underupdated_atwith the max value.test_created_at_state_restarts_from_start_date— the pre-upgrade state shape is ignored and the sync restarts fromstart_date(this is what makes the change breaking).test_notes_request_is_unbounded,test_boundary_date_note_is_not_dropped,test_legacy_date_state_is_accepted_and_emits_iso_statecarry over from the base PR.poe test-unit-tests: 5 passed.poe test-integration-tests: 2 passed, 6 skipped (this connector has noacceptance-test-config.ymlscenarios — unchanged here).No live Granola credentials were retrievable in this environment (
SECRET_SOURCE-GRANOLA__CREDS), so the API behavior in the mock comes from the reproduction in the issue and the vendor OpenAPI spec, not from a sync I ran. A prerelease validated against a real connection before the breaking release would be worthwhile.User Impact
Notes edited after creation — which is most of them, since Granola writes the summary after the meeting — are replicated on subsequent incremental syncs instead of staying frozen at their first-synced state. In exchange, existing connections must clear the
notesstream and re-sync once when upgrading.Can this PR be safely reverted and rolled back?
Rolling back returns the cursor key to
created_at, so state written by 0.3.0 would be ignored and the stream would restart fromstart_dateagain. A rollback needs the same clear-and-resync treatment as the upgrade.Link to Devin session: https://app.devin.ai/sessions/ce7406d5bb334fc5a23acbbc19c0db2d