Skip to content

Add max column config and session properties to equality delete as jo…#25462

Merged
tdcmeehan merged 1 commit intoprestodb:masterfrom
tdcmeehan:iceberg-eq-fix
Jul 2, 2025
Merged

Add max column config and session properties to equality delete as jo…#25462
tdcmeehan merged 1 commit intoprestodb:masterfrom
tdcmeehan:iceberg-eq-fix

Conversation

@tdcmeehan
Copy link
Contributor

@tdcmeehan tdcmeehan commented Jun 30, 2025

…in optimizer

Description

Adds session and config properties to the Iceberg connector allowing users to toggle the Iceberg delete-as-optimize rule based on the column count of the delete file. Deprecates the old binary flag, in the future it will be deleted and replaced with the column count toggle.

Motivation and Context

Fixes #25455 by introducing an upper bound to the number of columns allowed to use this optimization.

Impact

Fix #25455

Test Plan

Added unit tests demonstrating the toggling of the optimization.

Contributor checklist

  • Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

  == RELEASE NOTES ==

  Iceberg Connector Changes
  * Add ``iceberg.delete-as-join-rewrite-max-delete-columns`` configuration property and ``delete_as_join_rewrite_max_delete_columns`` session property to control when equality
  delete as join optimization is applied. The optimization is now only applied when the number of equality delete columns is less than or equal to this threshold (default: 400).
  Setting this to 0 disables the optimization. See :doc:`/connector/iceberg` for details.
  * Deprecate ``iceberg.delete-as-join-rewrite-enabled`` configuration property and ``delete_as_join_rewrite_enabled`` session property. Use
  ``iceberg.delete-as-join-rewrite-max-delete-columns`` instead.

@prestodb-ci prestodb-ci added the from:IBM PR from IBM label Jun 30, 2025
@prestodb-ci prestodb-ci requested review from a team, Joe-Abraham and Mariamalmesfer and removed request for a team June 30, 2025 15:14
@tdcmeehan tdcmeehan force-pushed the iceberg-eq-fix branch 4 times, most recently from f35ec66 to 4c66ee5 Compare June 30, 2025 16:02
@tdcmeehan tdcmeehan requested a review from jasonf20 June 30, 2025 16:04
Copy link
Contributor

@steveburnett steveburnett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Formatting suggestions. The text itself reads fine, no concern with the content.

@tdcmeehan
Copy link
Contributor Author

Thanks @steveburnett, I've incorporated your suggestions!

Copy link
Member

@hantangwangd hantangwangd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this change, lgtm overall. Just some little nits.

steveburnett
steveburnett previously approved these changes Jul 1, 2025
Copy link
Contributor

@steveburnett steveburnett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! (docs)

Pull updated branch, new local doc build. Looks good, thanks!

Copy link
Member

@hantangwangd hantangwangd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix, lgtm!

@tdcmeehan tdcmeehan merged commit 0eb6303 into prestodb:master Jul 2, 2025
118 of 119 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:IBM PR from IBM

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stack overflow when using IcebergEqualityDeleteAsJoin with wide tables and all-column equality deletes

4 participants