-
Notifications
You must be signed in to change notification settings - Fork 3k
REST: Set table format version for create table transactions #5084
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
kbendick
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is great. Thank you @bryanck!
Left one comment.
We should add a test as well, ideally / likely within CatalogTests, that when a V2 table is created we get a V2 table on load.
| private static List<MetadataUpdate> createChanges(TableMetadata meta) { | ||
| ImmutableList.Builder<MetadataUpdate> changes = ImmutableList.builder(); | ||
|
|
||
| changes.add(new MetadataUpdate.UpgradeFormatVersion(meta.formatVersion())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this should be set only if the format version is not the default?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the server-side default and the client-side default may not match? The issue I originally encountered was the Trino plugin had v2 as the default and set the initial sequence number to 1, but the server didn't receive a format version so treated it as v1, and failed because the sequence number was > 0. If the client and server are on different Iceberg versions, there could potentially be a mismatch in the default.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, I see what you mean.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1.
I could have sworn I left an almost identical comment when I reviewed this before (the single comment I was referring to). I can possibly help with adding a unit test for this sequence in CatalogTests if we’d like. Feel free to reach out on Slack too.
I too see what you mean.
kbendick
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me. Thank you @bryanck!
|
I added a test for creating a V2 table via a create transaction, which fails without the change. |
This PR adds the table format version in the list of metadata updates when starting a create table transaction with the REST catalog client.