·
3 commits
to master
since this release
Release 0.5.0.Beta
Breaking Changes
Debezium version upgraded from 2.5.x to 2.7.x:
- For Debezium 2.6 Breaking Changes please see release notes
- For Debezium 2.7 Breaking Changes please see release notes
IcebergSchemaHistory wrong table location corrected:
IcebergSchemaHistory class was using wrong table namespace {catalog-name}.{table-name} now this is corrected to use {table-namespace}.{table-name}
If you were using different catalog-name please follow following steps for migration
- stop debezium server
- copy the IcebergSchemaHistory table to new location:
{table-namespace}.{table-name} - start debezium server
Configuration change:
Configuration option debezium.sink.iceberg.upsert-op-column, defaultValue = __op is removed.
Notable changes
- Quarkus upgraded from 3.6.x to 3.10.x, See Quarkus release notes here
- Java release target upgraded from 11 to 17
- Full support for debezium nested events added. Now its possible to run consumer without event flattening.
- Debezium event processing improved, unnecessary schema de-serialization is avoided.
What's Changed
- Upgrade java to 17 by @ismailsimsek in #318
- Upgrade debezium to 2.7.0.Alpha1 by @ismailsimsek in #317
- Bump version.quarkus from 3.6.9 to 3.10.0 by @dependabot in #313
- Simplify example config, removed hadoop catalog example by @ismailsimsek in #321
- Minor, Improve naming of state tables used in tests by @ismailsimsek in #322
- Add mongodb flattening settings to example config by @ismailsimsek in #324
- Fix reading config values in debezium storage classes by @ismailsimsek in #327
- Minor improvements to icebergevents consumer, IcebergEventsChangeCons… by @ismailsimsek in #328
- Update codeql-analysis.yml to use java 17 by @ismailsimsek in #329
- Upgrade spark to 4.0.x version and enable Mysql tests by @ismailsimsek in #331
- Remove DynamicBatchSizeWait class by @ismailsimsek in #332
- Enforce maven version is greater than 3.8.6 by @ismailsimsek in #334
- Don't partition tables on append mode. by @ismailsimsek in #340
- Bump io.smallrye:jandex-maven-plugin from 3.1.7 to 3.2.0 by @dependabot in #338
- Bump org.postgresql:postgresql from 42.7.2 to 42.7.3 by @dependabot in #337
- Bump version.testcontainers from 1.19.7 to 1.19.8 by @dependabot in #336
- Improve IcebergChangeEvent class, Optimize event and schema deserialization by @ismailsimsek in #343
- Fail deduplication if event key is null by @ismailsimsek in #344
- Improve naming, rename JsonSchema to ChangeEventSchema by @ismailsimsek in #345
- Improve code and schema conversion tests by @ismailsimsek in #346
- Add more tests on schema conversion by @ismailsimsek in #348
- Improve conversion of debezium schema to iceberg schema by @ismailsimsek in #347
- Revert "Bump io.smallrye:jandex-maven-plugin from 3.1.7 to 3.2.0 (#338)" by @ismailsimsek in #349
- Improve setting identifier fields by @ismailsimsek in #350
- Allow nested fields to be set as identifier by @ismailsimsek in #351
- Lower minimum maven version enforcement to 3.6.3 by @ismailsimsek in #353
- Optimize test logs, decrease verbose logs by @ismailsimsek in #354
- Check if configuration contains event flattening by @ismailsimsek in #352
- Test Field Addition is working, automatic schema expansion by @ismailsimsek in #355
- Test consumer without event flattening, Add flag to control identifier-field creation by @ismailsimsek in #356
- Document new features and improve documentation by @ismailsimsek in #357
- Remove stage repo used for spark 4.0.0-preview1 dependency by @ismailsimsek in #358
- Update README.md by @ismailsimsek in #359
- Update dependabot to weekly interval by @ismailsimsek in #360
- Bump docker/build-push-action from 5 to 6 by @dependabot in #361
- Improve handling delete events and logging message by @ismailsimsek in #366
- Improve documentation, schema change behaviour by @ismailsimsek in #367
- Bump com.squareup.okhttp3:okhttp from 4.11.0 to 4.12.0 by @dependabot in #364
- Bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.5 to 3.3.0 by @dependabot in #363
- Bump version.debezium from 2.7.0.Alpha1 to 2.7.0.Final by @dependabot in #371
Full Changelog: 0.4.0.Final...0.5.0.Beta
Contributors
ismailsimsek and dependabot