feat(plugin-iceberg): Add DDL statements to drop branches and tags by agrawalreetika · Pull Request #23614 · prestodb/presto

agrawalreetika · 2024-09-10T15:17:54Z

Description

Add DDL statements to drop branches and tags

Motivation and Context

Resolves #22028

Impact

Resolves #22028

SQL support for dropping a branch from a table :

ALTER TABLE users DROP BRANCH 'branch1';

SQL support for dropping a tag from a table :

ALTER TABLE users DROP TAG 'tag1';

Test Plan

Contributor checklist

Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
Documented new properties (with its default value), SQL syntax, functions, or other functionality.
If release notes are required, they follow the release notes guidelines.
Adequate tests were added if applicable.
CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* Add DDL support for dropping a branch from a table. 
* Add DDL support for dropping a tag from a table.

Iceberg Connector Changes
* Add support for dropping a branch from an Iceberg table.
* Add support for dropping a tag from an Iceberg table.

github-actions · 2024-09-10T15:18:54Z

Codenotify: Notifying subscribers in CODENOTIFY files for diff 367b7d7...84553ba.

Notify	File(s)
@aditi-pandit	presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4
@elharo	presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4
@kaikalur	presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4
@rschlussel	presto-parser/src/main/antlr4/com/facebook/presto/sql/parser/SqlBase.g4

steveburnett

LGTM! (docs)

Pull branch, local docs build, looks good. Thanks!

ZacBlanco

Some high level feedback. Most of the code looks good. I will take a closer look on 2nd pass.

Also, I wanted to bring a thought I had about parser extensions. Since many connectors will not support this branching and tagging, I was thinking that maybe we ought to consider designing a SQL syntax plugin extension interface. Spark allows custom syntax extensions through implementing some set of interfaces or bringing your own parser. The upstream iceberg project now maintains their own spark SQL syntax extensions.

I'm not proposing we need that for this PR, but maybe it's something we should start thinking about if connectors start adding more radically different features that would be best left to some syntax extensions/optional plugins, especially for things outside the SQL specification.

ZacBlanco · 2024-09-11T03:14:03Z

presto-hive/src/main/java/com/facebook/presto/hive/security/SqlStandardAccessControl.java

+    }
+
+    @Override
+    public void checkCanDropTag(ConnectorTransactionHandle transaction, ConnectorIdentity identity, AccessControlContext context, SchemaTableName tableName)


I wonder about the granularity of these methods. In other implementation (e.g. spark?) at what granularity do they enforce the ability to do CRUD operations on tags and branches?

I'm thinking about a few cases

A group or user(s) can only access a certain set of branches or tags

A group or user(s) can only create branches starting from a specific branch

A group or user(s) can create tags

I know we're only implementing DROP but I want to understand the whole story for access control around branches and tags. Would we ever need to pass the branch/tag to these methods?

I think the granularity of access control methods in Spark for CRUD operations on Iceberg tags and branches is limited by Spark's integration with external systems (such as file systems, catalogs, and security frameworks like Apache Ranger).
For example, Ranger policies can define access controls at the table level, which could be extended to manage specific branches or tag-based access.
And like for cloud-based catalogs like AWS Glue, you can control access to Iceberg metadata (branches and tags) via IAM policies that grant or restrict specific operations.

Thanks for the info. Would the parameters passed here as context have enough information for us to act at a similar granularity? I don't see anything in the method parameters that contains the branch name which I assume we would need to perform access control at a similar level.

I would like to discuss around this, Systems like Ranger can define access controls at the table level, column level. So in this case I think access of drop branch & tags could be table based. As per I can think of branch & tag level policies then has to be maintained on engine side if we introduce branch name / tag name in here?

@tdcmeehan What do you think about access control for branches and tags? Would be based on the parent table itself or based on the tags/branches?
My thinking was that since no policies are enforced based on branch/tags via security frameworks, should this honor the same access policies as table? Or if we don't even need access control exposed for dropTag & dropBranch?

presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergAbstractMetadata.java

presto-spi/src/main/java/com/facebook/presto/spi/connector/ConnectorMetadata.java

presto-iceberg/src/test/java/com/facebook/presto/iceberg/IcebergDistributedTestBase.java

agrawalreetika · 2024-09-13T04:57:43Z

@ZacBlanco Can you please take another pass?

hantangwangd

The whole change overall looks good to me. Some little nits, and one problem for discussing about the behavior of if exists on branch and tag.

presto-parser/src/main/java/com/facebook/presto/sql/tree/DropBranch.java

presto-parser/src/main/java/com/facebook/presto/sql/tree/DropTag.java

presto-main/src/main/java/com/facebook/presto/execution/DropTagTask.java

presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergAbstractMetadata.java

presto-iceberg/src/test/java/com/facebook/presto/iceberg/IcebergDistributedTestBase.java

agrawalreetika · 2024-09-25T07:11:22Z

@hantangwangd Thanks for you review. I have addressed your comments, please review.

hantangwangd

Thanks for the fix. LGTM!

agrawalreetika · 2024-09-27T07:47:05Z

Hi @tdcmeehan, This is now waiting for the final committer review. Please review whenever you have time.

pratyakshsharma

Thank you for the PR, minor comments, and rest looks good.

pratyakshsharma · 2024-11-06T11:09:58Z

presto-main/src/main/java/com/facebook/presto/execution/DropBranchTask.java

+
+        Optional<MaterializedViewDefinition> optionalMaterializedView = metadata.getMetadataResolver(session).getMaterializedView(tableName);
+        if (optionalMaterializedView.isPresent()) {
+            if (!statement.isTableExists()) {


should we throw an error in cases when if exists is present in the query and it is a materialized view? If drop branch is just not supported for MVs, ideally error should be thrown irrespective of exists check?

Alternate way of handling this can be introducing an enum Type in DropBranch and have only TABLE as the supported type.

pratyakshsharma · 2024-11-06T11:12:26Z

presto-main/src/main/java/com/facebook/presto/execution/DropBranchTask.java

+            return immediateFuture(null);
+        }
+
+        ConnectorId connectorId = metadata.getCatalogHandle(session, tableName.getCatalogName())


nit: the variable connectorId is not getting used and can be removed. Also I have refactored and extracted this piece of code in MetadataUtil class in this commit - dec2952.
So we can probably do a similar change here as well so that all the Task classes use the same methods?i

pratyakshsharma · 2024-11-06T11:13:29Z

presto-main/src/main/java/com/facebook/presto/execution/DropTagTask.java

+            return immediateFuture(null);
+        }
+
+        ConnectorId connectorId = metadata.getCatalogHandle(session, tableName.getCatalogName())


nit: same as above.

pratyakshsharma · 2024-11-06T11:14:00Z

presto-main/src/main/java/com/facebook/presto/execution/DropTagTask.java

+
+        Optional<MaterializedViewDefinition> optionalMaterializedView = metadata.getMetadataResolver(session).getMaterializedView(tableName);
+        if (optionalMaterializedView.isPresent()) {
+            if (!statement.isTableExists()) {


nit: same as above. What about the case when if exists is present and it is MV?

steveburnett · 2025-04-24T19:00:27Z

Thanks for the release note! Nit on phrasing to follow the Order of changes in the Release Notes Guidelines.

== RELEASE NOTES ==

General Changes
* Add DDL support for dropping a branch from a table. 
* Add DDL support for dropping a tag from a table.

Iceberg Connector Changes
* Add support for dropping a branch from an Iceberg table.
* Add support for dropping a branch from an Iceberg table.

hantangwangd

Thanks @agrawalreetika, LGTM for this change. +1 to @ZacBlanco's point about the extension mechanism for connector-specific SQL syntax - maybe we should start looking into this.

tdcmeehan · 2025-12-03T21:14:18Z

I'm comfortable with the idea of this change. While we should be cautious around the standard eventually adopting this type of syntax, the pace at which is the standard moves is far slower than even the Iceberg spec moves. It is too slow for us to block progress on this PR.

I agree with the idea of customizing the parser and planning in general though, along the same lines as Spark. However, given we have already broken standards compliance with time travel (on SYSTEM_VERSION), I'm comfortable pushing ahead with this.

hantangwangd

@agrawalreetika thanks for this feature, lgtm! Just a heads-up: the two entries under Iceberg Connector Changes appear to be duplicates.

agrawalreetika requested review from a team, ZacBlanco, elharo, feilong-liu, hantangwangd, jaystarshot, steveburnett and vivek-bharathan as code owners September 10, 2024 15:17

agrawalreetika requested a review from presto-oss September 10, 2024 15:17

agrawalreetika self-assigned this Sep 10, 2024

steveburnett previously approved these changes Sep 10, 2024

View reviewed changes

tdcmeehan self-assigned this Sep 10, 2024

agrawalreetika dismissed steveburnett’s stale review via ee54cd3 September 10, 2024 19:20

agrawalreetika force-pushed the iceberg-tag-branch-drop branch from 3fe4d56 to ee54cd3 Compare September 10, 2024 19:20

ZacBlanco requested changes Sep 11, 2024

View reviewed changes

agrawalreetika force-pushed the iceberg-tag-branch-drop branch 2 times, most recently from 8f4d3fa to 1e47c05 Compare September 11, 2024 21:18

hantangwangd requested changes Sep 17, 2024

View reviewed changes

agrawalreetika force-pushed the iceberg-tag-branch-drop branch 3 times, most recently from 0abecd5 to 5af0a0d Compare September 23, 2024 18:54

hantangwangd previously approved these changes Sep 25, 2024

View reviewed changes

ZacBlanco previously approved these changes Sep 25, 2024

View reviewed changes

pratyakshsharma reviewed Nov 6, 2024

View reviewed changes

agrawalreetika dismissed stale reviews from ZacBlanco and hantangwangd via 260b490 February 7, 2025 19:34

agrawalreetika force-pushed the iceberg-tag-branch-drop branch from 5af0a0d to 260b490 Compare February 7, 2025 19:34

agrawalreetika force-pushed the iceberg-tag-branch-drop branch from 260b490 to e333bb1 Compare May 5, 2025 05:19

agrawalreetika force-pushed the iceberg-tag-branch-drop branch from e333bb1 to dbabe20 Compare May 14, 2025 07:56

hantangwangd previously approved these changes May 16, 2025

View reviewed changes

agrawalreetika dismissed hantangwangd’s stale review via 10d47a5 November 19, 2025 05:09

agrawalreetika force-pushed the iceberg-tag-branch-drop branch from dbabe20 to 10d47a5 Compare November 19, 2025 05:09

agrawalreetika changed the title ~~Add DDL statements to drop branches and tags~~ feat(plugin-iceberg): Add DDL statements to drop branches and tags Nov 19, 2025

agrawalreetika force-pushed the iceberg-tag-branch-drop branch 2 times, most recently from 9522908 to d91e3ae Compare November 19, 2025 08:19

agrawalreetika added 4 commits December 3, 2025 14:26

Add DROP BRANCH sql support

9ac432c

Add support for drop branch for Iceberg Table

a51f418

Add DROP TAG sql support

0800725

Add support for drop tag for Iceberg Table

84553ba

agrawalreetika force-pushed the iceberg-tag-branch-drop branch from d91e3ae to 84553ba Compare December 3, 2025 08:57

hantangwangd approved these changes Dec 10, 2025

View reviewed changes

tdcmeehan approved these changes Dec 10, 2025

View reviewed changes

agrawalreetika merged commit a2c2b43 into prestodb:master Dec 11, 2025
83 checks passed

agrawalreetika deleted the iceberg-tag-branch-drop branch December 11, 2025 04:17

This was referenced Mar 31, 2026

docs: Add release notes for 0.297 unix280/presto#51

Closed

docs: Add release notes for 0.297 unix280/presto#52

Open

prestodb-ci mentioned this pull request Apr 1, 2026

docs: Add release notes for 0.297 #27484

Open

15 tasks

Conversation

agrawalreetika commented Sep 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Impact

Test Plan

Contributor checklist

Release Notes

Uh oh!

github-actions bot commented Sep 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steveburnett left a comment

Choose a reason for hiding this comment

Uh oh!

ZacBlanco left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ZacBlanco Sep 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agrawalreetika Sep 11, 2024

Choose a reason for hiding this comment

Uh oh!

ZacBlanco Sep 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agrawalreetika Sep 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agrawalreetika Sep 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

agrawalreetika commented Sep 13, 2024

Uh oh!

hantangwangd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

agrawalreetika commented Sep 25, 2024

Uh oh!

hantangwangd left a comment

Choose a reason for hiding this comment

Uh oh!

agrawalreetika commented Sep 27, 2024

Uh oh!

pratyakshsharma left a comment

Choose a reason for hiding this comment

Uh oh!

pratyakshsharma Nov 6, 2024

Choose a reason for hiding this comment

Uh oh!

pratyakshsharma Nov 6, 2024

Choose a reason for hiding this comment

Uh oh!

pratyakshsharma Nov 6, 2024

Choose a reason for hiding this comment

Uh oh!

agrawalreetika commented Sep 10, 2024 •

edited

Loading

github-actions bot commented Sep 10, 2024 •

edited

Loading

ZacBlanco left a comment •

edited

Loading

ZacBlanco Sep 11, 2024 •

edited

Loading

ZacBlanco Sep 11, 2024 •

edited

Loading

agrawalreetika Sep 11, 2024 •

edited

Loading

agrawalreetika Sep 13, 2024 •

edited

Loading