Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Throw TrinoException when Iceberg commit fails #23312

Conversation

pajaks
Copy link
Member

@pajaks pajaks commented Sep 6, 2024

Description

In case Iceberg commit fails throw TrinoException with clear error type.

Improvement for #23008 where only Transaction.commitTransaction() was considered. IcebergUtil.commit() can throw the same exception.

Addresses also flakiness of testDeleteRowsConcurrently and testUpdateRowConcurrently Iceberg tests related to type and message of exception.

Additional context and related issues

Example stack trace:

2024-09-05T16:32:38.8843159Z Caused by: org.apache.iceberg.exceptions.ValidationException: Found conflicting files that can contain records matching true: [s3://test-bucket-a94uh2d33c/tpch/test_concurrent_updatevwdljsy51o-6bb6f3a9a4d343798b94bd88ba4f52c1/data/20240905_162843_02756_a284y-89d536c1-54b9-4d5e-aa19-dc2154d5c5e8.parquet]
2024-09-05T16:32:38.8851856Z 	at org.apache.iceberg.MergingSnapshotProducer.validateAddedDataFiles(MergingSnapshotProducer.java:347)
2024-09-05T16:32:38.8858939Z 	at org.apache.iceberg.BaseRowDelta.validate(BaseRowDelta.java:130)
2024-09-05T16:32:38.8864641Z 	at org.apache.iceberg.SnapshotProducer.apply(SnapshotProducer.java:235)
2024-09-05T16:32:38.8870758Z 	at org.apache.iceberg.SnapshotProducer.lambda$commit$2(SnapshotProducer.java:386)
2024-09-05T16:32:38.8876812Z 	at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
2024-09-05T16:32:38.8882044Z 	at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219)
2024-09-05T16:32:38.8885484Z 	at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203)
2024-09-05T16:32:38.8888431Z 	at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196)
2024-09-05T16:32:38.8892052Z 	at org.apache.iceberg.SnapshotProducer.commit(SnapshotProducer.java:384)
2024-09-05T16:32:38.8895306Z 	at io.trino.plugin.iceberg.IcebergUtil.commit(IcebergUtil.java:979)

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

@github-actions github-actions bot added the iceberg Iceberg connector label Sep 6, 2024
@pajaks pajaks force-pushed the pajaks/iceberg_validation_exception_handling branch from b3e4cb9 to 0fd53b8 Compare September 6, 2024 09:46
@pajaks pajaks force-pushed the pajaks/iceberg_validation_exception_handling branch from 0fd53b8 to b4c7f28 Compare September 6, 2024 11:16
@raunaqmorarka raunaqmorarka merged commit a071a79 into trinodb:master Sep 9, 2024
42 checks passed
@github-actions github-actions bot added this to the 458 milestone Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed iceberg Iceberg connector
Development

Successfully merging this pull request may close these issues.

4 participants