-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Spec: Make next-row-id required in v3 #12757
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Do we need to notify the dev list? |
|
@manuzhang It's already notified at https://lists.apache.org/thread/61w6lj3tmnyv1gtvmz08twbp6lb8nzsx |
|
Okay, this is a follow-up of #12580. |
|
Hold on, I'm not certain this is required. When a table is first converted to |
|
I would hold off on this for a little bit. I'm looking more deeply into v1/v2 conversion to v3 and I think Dan is probably right. We will very likely have a case where the next-row-id of the table is not yet known just after the conversion. So even though the metadata file is v3, before the next version writes a new snapshot (and manifest list) the number of existing rows is not precisely known. I'll update this when I know more since I'm actively working on it. |
|
After working on the implementation and posting a PR to update the spec for how we handle upgrades, I think that this PR is correct and that +1 to making the field required. |
|
I'm wondering whether we have a test to verify the upgrade path. |
|
@ebyhr, I cherry-picked this commit into #12781 with the other spec changes that I just raised a vote for on the dev list so that we have this specifically included in a vote. When we go to merge, I can either merge this PR and then mine or make sure you're a co-author of the other PR. Whatever you'd prefer. |
There are extensive tests in #12672. See TestRowLineageAssignment. |
|
Closing as #12781 has been merged. Thank you for the cherry-pick. |
|
Merged as part of #12781. |
#12593 made
next-row-idfield required in my understanding.