[bulk-extract-cdk] skip errors caused by unparseable DDLs #48612
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
|
|
| @@ -57,6 +57,8 @@ class DebeziumPropertiesBuilder(private val props: Properties = Properties()) { | |||
| with("value.converter.replace.null.with.default", "false") | |||
| // Timeout for DebeziumEngine's close() method. | |||
| with("debezium.embedded.shutdown.pause.before.interrupt.ms", "10000") | |||
| // Unblock CDC syncs by skipping errors caused by unparseable DDLs | |||
| with("schema.history.internal.skip.unparseable.ddl", "true") | |||
There was a problem hiding this comment.
This is indeed the correct Debezium property, but by adding these lines of code here you're setting this property for ALL connectors. Not all connectors maintain a schema history file. That being said, this is just ignored for those connectors, and it seems like this property is named the same way for all those that do maintain a schema history.
There was a problem hiding this comment.
this is a good point, currently do we have a way for all the connectors to share a common set of debezium properties, while allows specific connectors to include extra properties that won't be shared?
|
@postamar I see this fix is sent to new cdk, so do we also need to bump up the version of the CDK? |
theyueli
changed the title
What
This change is to fix debezium bug mentioned in #48497
How
It fixes the problem by setting schema.history.internal.skip.unparseable.ddl in DebeziumProperties to True to skip errors caused by unparseable DDLs
Review guide
Reproduced the problem locally
Before the fix:
After fix:
User Impact
None
Can this PR be safely reverted and rolled back?