Skip to content

[Source-postgres] : Bump cdk to latest #39474

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
akashkulk merged 9 commits into from
Jun 14, 2024
Merged

[Source-postgres] : Bump cdk to latest #39474

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
cf54dbc
Bump cdks
akashkulk Jun 13, 2024
66b444c
Fixes
akashkulk Jun 13, 2024
b20d850
Merge branch 'master' into akash/bump-dbz-interface-change
akashkulk Jun 13, 2024
5a90f55
Merge branch 'master' into akash/bump-dbz-interface-change
akashkulk Jun 13, 2024
cf7bfcc
revert mssql
akashkulk Jun 14, 2024
ff0975f
Merge branch 'master' into akash/bump-dbz-interface-change
akashkulk Jun 14, 2024
c61bac2
Merge branch 'master' into akash/bump-dbz-interface-change
akashkulk Jun 14, 2024
8bca171
Merge branch 'master' into akash/bump-dbz-interface-change
akashkulk Jun 14, 2024
6ec3c74
Merge branch 'master' into akash/bump-dbz-interface-change
akashkulk Jun 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-postgres/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ java {
}

airbyteJavaConnector {
cdkVersionRequired = '0.36.5'
cdkVersionRequired = '0.38.1'
features = ['db-sources', 'datastore-postgres']
useLocalCdk = false
}
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-postgres/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ data:
connectorSubtype: database
connectorType: source
definitionId: decd338e-5647-4c0b-adf4-da0e75f5a750
dockerImageTag: 3.4.15
dockerImageTag: 3.4.16
dockerRepository: airbyte/source-postgres
documentationUrl: https://docs.airbyte.com/integrations/sources/postgres
githubIssueLabel: source-postgres
Expand Down
5 changes: 4 additions & 1 deletion ...src/main/java/io/airbyte/integrations/source/postgres/cdc/PostgresCdcCtidInitializer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -240,8 +240,11 @@ public static List<AutoCloseableIterator<AirbyteMessage>> cdcCtidIteratorsCombin
final AirbyteDebeziumHandler<Long> handler = new AirbyteDebeziumHandler<>(sourceConfig,
targetPosition, false, firstRecordWaitTime, queueSize, false);
final PostgresCdcStateHandler postgresCdcStateHandler = new PostgresCdcStateHandler(stateManager);
final var cdcStreamList = catalog.getStreams().stream()
.filter(stream -> stream.getSyncMode() == SyncMode.INCREMENTAL)
.map(stream -> stream.getStream().getNamespace() + "." + stream.getStream().getName()).toList();
final var propertiesManager = new RelationalDbDebeziumPropertiesManager(
PostgresCdcProperties.getDebeziumDefaultProperties(database), sourceConfig, catalog);
PostgresCdcProperties.getDebeziumDefaultProperties(database), sourceConfig, catalog, cdcStreamList);
final var eventConverter = new RelationalDbDebeziumEventConverter(new PostgresCdcConnectorMetadataInjector(), emittedAt);

final Supplier<AutoCloseableIterator<AirbyteMessage>> incrementalIteratorSupplier = () -> handler.getIncrementalIterators(
Expand Down
3 changes: 2 additions & 1 deletion .../src/main/java/io/airbyte/integrations/source/postgres/cdc/PostgresDebeziumStateUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,8 @@ public OptionalLong savedOffset(final Properties baseProperties,
final JsonNode cdcState,
final JsonNode config) {
final var offsetManager = AirbyteFileOffsetBackingStore.initializeState(cdcState, Optional.empty());
final DebeziumPropertiesManager debeziumPropertiesManager = new RelationalDbDebeziumPropertiesManager(baseProperties, config, catalog);
final DebeziumPropertiesManager debeziumPropertiesManager =
new RelationalDbDebeziumPropertiesManager(baseProperties, config, catalog, Collections.emptyList());
final Properties debeziumProperties = debeziumPropertiesManager.getDebeziumProperties(offsetManager);
return parseSavedOffset(debeziumProperties);
}
Expand Down
3 changes: 2 additions & 1 deletion docs/integrations/sources/postgres.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,8 @@ According to Postgres [documentation](https://www.postgresql.org/docs/14/datatyp

| Version | Date | Pull Request | Subject |
|---------|------------|----------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 3.4.15 | 2024-05-29 | [38773](https://github.com/airbytehq/airbyte/pull/38773) | Connect with adaptiveFetch=true. |
| 3.4.16 | 2024-05-29 | [39474](https://github.com/airbytehq/airbyte/pull/39474) | Adopt latest CDK. |
| 3.4.15 | 2024-05-29 | [38773](https://github.com/airbytehq/airbyte/pull/38773) | Connect with adaptiveFetch=true. |
| 3.4.14 | 2024-06-08 | [39353](https://github.com/airbytehq/airbyte/pull/39353) | Upgrade Debezium to 2.6.2 |
| 3.4.13 | 2024-06-04 | [38875](https://github.com/airbytehq/airbyte/pull/38875) | read() throws config exception upon detecting transaction ID wraparound. |
| 3.4.12 | 2024-06-04 | [38836](https://github.com/airbytehq/airbyte/pull/38836) | check() throws config error upon detecting transaction ID wraparound. |
Expand Down