-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-29517][SQL] TRUNCATE TABLE should look up catalog/table like v2 commands #26174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Test build #112312 has finished for PR 26174 at commit
|
imback82
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
| v1TableName.asTableIdentifier, | ||
| "MSCK REPAIR TABLE") | ||
|
|
||
| case TruncateTableStatement(tableName, partitionSpec) => |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shall we add a truncateTable method to TableCatalog? cc @rdblue
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No. This is not a catalog operation, it is a table operation. We already have 2 ways to truncate a table: if the table implements SupportsTruncate or SupportsOverwrite. A truncate command should build a write and commit without any commit messages.
We can also add a short-cut trait on Table to avoid the builder.
|
Looks good to me if we don't want to add the implementation in the same PR. |
|
I can add the V2 part in this PR or separate one. @cloud-fan |
|
Since it's unclear how to implement a v2 truncate table, let's leave it first. |
|
thanks, merging to master! |
How is this unclear? There is a truncate API in v2. |
|
I was referring to It's unclear if we should submit a job to do truncate, or add a short-cut trait. |
|
Even if we add a short-cut trait, Spark should fall back the the builder method if the table supports it. |
What changes were proposed in this pull request?
Add TruncateTableStatement and make TRUNCATE TABLE go through the same catalog/table resolution framework of v2 commands.
Why are the changes needed?
It's important to make all the commands have the same table resolution behavior, to avoid confusing end-users. e.g.
Does this PR introduce any user-facing change?
yes. When running TRUNCATE TABLE, Spark fails the command if the current catalog is set to a v2 catalog, or the table name specified a v2 catalog.
How was this patch tested?
Unit tests.