Skip to content

Conversation

@rdblue
Copy link
Contributor

@rdblue rdblue commented Aug 7, 2020

This PR updates classes that use StructLikeWrapper to correctly track partitions using both ID and tuple, using DataFile.specId from #1317 and PartitionSet added by #1307.

This is needed to correctly track partitions because tuples can be valid for more than one spec. For example, (12974) is a valid partition for both hours(ts) and days(ts) partition specs, but does not represent the same logical partition in both.

@rdblue rdblue force-pushed the fix-partition-sets branch from 605c87b to 6d8a60e Compare August 10, 2020 22:58
@rdblue rdblue mentioned this pull request Aug 11, 2020
@rdblue rdblue force-pushed the fix-partition-sets branch from 81b70fc to b136822 Compare August 11, 2020 23:15
@rdblue rdblue force-pushed the fix-partition-sets branch from b136822 to daacf7b Compare August 11, 2020 23:17
}
this.hashCode = result;
}
this.hashCode = structHash.hash(struct);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shardulm94, this removes the temporary work-arounds in StructLikeWrapper.

@rdblue rdblue requested a review from shardulm94 August 11, 2020 23:19
Copy link
Contributor

@shardulm94 shardulm94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@rdblue rdblue merged commit 01986ea into apache:master Aug 12, 2020
@rdblue
Copy link
Contributor Author

rdblue commented Aug 12, 2020

Thanks for reviewing, @shardulm94!

cmathiesen pushed a commit to ExpediaGroup/iceberg that referenced this pull request Aug 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants