-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"java.lang.RuntimeException: Failed to convert timestamp value, field: __source_ts_ms value: 0" with MongoDB source #291
Comments
@julianpark90 could you please share stack trace and your config? |
@ismailsimsek The provided includes my configuration and the error trace. The issue occurs only when the program runs with pre-existing data in MongoDB, but a ts_ms value of 0 should not cause this problem, I guess.
I've encountered another problem directly linked to issue DBZ-6725, PR. I'm currently using MongoDB versions 4.4 and 5.0, and upgrading to version 6.0 is not feasible for me. Here's the stack trace for reference.
Also I've tried with other options(debezium.sink.iceberg.upsert-keep-deletes=false) than rewrite mode then I get the below.
Is there a possible workaround for this issue by adjusting configurations? Any help would be greatly appreciated. |
@julianpark90 latest release now handles as far as i see for the delete events your best option is to drop the, ignore them. or use version 6.0 |
@ismailsimsek Thank you for the patch. It seems to be working fine. As for the mongodb deletion, what if we do something like
Lines 70 to 72 in 1558a79
so that if id is missing from the data to be written, in this case strictly for the mongodb deletion, it fixes the value by adding the id |
@julianpark90 I think this fix should be added to Debezium project, "payload": {
"id": 1004,
"__op": "d"
"__deleted": true
} |
@ismailsimsek, I faced the same problem about deleting events. With insert and update operation, the connector works well
If I set the configuration delete.tombstone.handling.mode to "drop" as you suggested, deleting events will be skipped. This could result in duplicated data in the target table. Is there any other way to handle this problem |
@duc-dn i believe this should be added to Debezium project. i don't see any other way. |
@duc-dn @julianpark90 For the delete events issue, i created ticket for debezium project https://issues.redhat.com/browse/DBZ-7695 feel free to comment or implement it.. FYI |
great, thanks @ismailsimsek |
I'm encountering a problem where the __source_ts_ms field from MongoDB consistently returns 0, resulting in the runtime exception.
Why does __source_ts_ms always return 0, and how can I confirm the validity of MongoDB timestamps?
Also, it's odd that a value of 0 fails to match isLong() or isTextual(), resulting in exceptions.
Any advice on managing the constant 0 in __source_ts_ms would be helpful.
The text was updated successfully, but these errors were encountered: