-
Notifications
You must be signed in to change notification settings - Fork 3k
Spark: Fix isIcebergCommand check and style issues from #6638 #6728
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
jackye1995
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh nice catch, did not realize this at all
.../scala/org/apache/spark/sql/catalyst/parser/extensions/IcebergSparkSqlExtensionsParser.scala
Show resolved
Hide resolved
71e3604 to
a7bcb44
Compare
|
Thanks for the fixes, let's definitely enable scala checkstyle asap to avoid issues like this. Also I think there are some other fixes in the tests that Liwei found, could you also fix those? And the title of the PR should probably change. |
a7bcb44 to
9aecd25
Compare
Done! Agreed I created #6736, to track this to avoid reviewers having to spend time on style issues in the scala codebase. I'm looking into it. Thanks for the reviews @jackye1995 @hililiwei @yyanyy |
|
Thanks, I will merge the change for now, as you are working on fixing Scala checkstyle, we can make any missing changes with that. |
@hililiwei @jackye1995 @flyrain @yyanyy When working on the drop branch implementation, I noticed that the "or" condition for replace branch wasn't placed right in #6638 . It should be within the startsWith("alter table") condition, but currently its outside. The Iceberg DDL will still work, but the check should be more specific to match the alter table condition to prevent any future spark sql syntax change from being bypassed in the
isIcebergCommandcheck.Just raising this PR separately since it's a bugfix (albeit for an unlikely case)