Spark 3.3: SQL Extensions for DROP TAG #6807
Conversation
Co-authored-by: Amogh Jahagirdar <[email protected]> Co-authored-by: chidayong <[email protected]>
amogh-jahagirdar
left a comment
amogh-jahagirdar
left a comment
There was a problem hiding this comment.
Mostly looks good to me thanks @hililiwei ! Just some test cases I think we can clean up
| Assert.assertEquals(first, ref.snapshotId()); | ||
| } | ||
|
|
||
| @Test |
There was a problem hiding this comment.
I think we should also have a test to ensure that DROP TAG does not work when the ref is a branch. If I missed the opposite case for DROP BRANCH, I think let's add that as well for completeness.
| AssertHelpers.assertThrows( | ||
| "Non-conforming tag name", | ||
| IcebergParseException.class, | ||
| "mismatched input '123'", | ||
| () -> sql("ALTER TABLE %s DROP TAG %s", tableName, "123")); |
There was a problem hiding this comment.
I think it's fine to include this test but could we separate it and have this test just focused on the happy case?
| sql("ALTER TABLE %s DROP TAG %s", tableName, tagName); | ||
| table.refresh(); | ||
| ref = table.refs().get(tagName); | ||
| Assert.assertNull(ref); |
There was a problem hiding this comment.
nit: we should prefer to add a description for all the assertions, there are many assertEquals and assertNull here that lack description
|
Overall looks good to me, just some nit comments about the tests |
....3/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/DropTag.scala
Outdated
Show resolved
Hide resolved
| | ALTER TABLE multipartIdentifier DROP BRANCH (IF EXISTS)? identifier #dropBranch | ||
| | ALTER TABLE multipartIdentifier DROP TAG (IF EXISTS)? identifier #dropTag |
There was a problem hiding this comment.
can this be written like :
| | ALTER TABLE multipartIdentifier DROP BRANCH (IF EXISTS)? identifier #dropBranch | |
| | ALTER TABLE multipartIdentifier DROP TAG (IF EXISTS)? identifier #dropTag | |
| | ALTER TABLE multipartIdentifier DROP (BRANCH | TAG) (IF EXISTS)? identifier #dropBranchOrTag |
There was a problem hiding this comment.
we discussed this in #6637 (comment), and the conclusion was that it is more clear to have them separated.
The current way for separated DROP BRANCH and DROP TAG makes the logic consistent with CREATE.
Let me know what you think!
There was a problem hiding this comment.
I see, was mostly coming from spark code base, considering this example :
If we have reached the concencus, then it should be fine.
…l/catalyst/plans/logical/DropTag.scala Co-authored-by: Prashant Singh <[email protected]>
|
Restart CI |
amogh-jahagirdar
left a comment
amogh-jahagirdar
left a comment
There was a problem hiding this comment.
Looks good to me! Thanks @hililiwei for the contribution!
singhpk234
left a comment
singhpk234
left a comment
There was a problem hiding this comment.
LGTM, Thanks @hililiwei !
| | ALTER TABLE multipartIdentifier DROP BRANCH (IF EXISTS)? identifier #dropBranch | ||
| | ALTER TABLE multipartIdentifier DROP TAG (IF EXISTS)? identifier #dropTag |
There was a problem hiding this comment.
I see, was mostly coming from spark code base, considering this example :
If we have reached the concencus, then it should be fine.
...k/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestBranchDDL.java
Outdated
Show resolved
Hide resolved
...k/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestBranchDDL.java
Outdated
Show resolved
Hide resolved
…park/extensions/TestBranchDDL.java Co-authored-by: Amogh Jahagirdar <[email protected]>
|
Looks like we have all comments addressed, I will go ahead to merge the PR. Thanks for the contribution @hililiwei , and thanks for everyone's review! |
|
Thank you all for review 😄 |
Co-authored-by: Amogh Jahagirdar <[email protected]> Co-authored-by: chidayong <[email protected]>
5 participants
Co-authored-by: Amogh Jahagirdar [email protected]
Co-authored-by: chidayong [email protected]