Fix struct comparison and add struct and partition set implementations #1307
Conversation
| } | ||
|
|
||
| static class PartitionSet { | ||
| static class PartitionMap { |
There was a problem hiding this comment.
This was renamed to avoid a conflict with the new PartitionSet class, and updated to use StructLikeWrapper with correct equals/hashCode implementations.
|
Let me take a look right now. |
aokolnychyi
left a comment
aokolnychyi
left a comment
There was a problem hiding this comment.
The change looks good to me, just few questions for my understanding.
How do we plan to test this out?
| if (structHash != null) { | ||
| this.hashCode = structHash.hash(struct); | ||
| } else { | ||
| int result = 97; |
There was a problem hiding this comment.
Is this the work-around when we don't have a type? In which case does this happen?
There was a problem hiding this comment.
Yes. This is a work-around for when there is no type, so that the current code paths don't fail. The follow-up commit removes this after adding specId to DataFile.
|
Thanks for reviewing, @aokolnychyi! Tests are passing now, so I'll merge. |
2 participants
StructLikeWrapperis used to provideequalsandhashCodemethods that are consistent across allStructLikeimplementations. But, the implementation did not handle different implementations ofCharSequencecorrectly. This fixes the implementations by adding aComparatorimplementation for structs, and adding classes to implement Java'shashCode. These implementations depend on the Iceberg type of an object.The type of a struct is not always passed to
StructLikeWrapper, so this includes a temporary work-around to use the existing implementation for code paths that do not currently have a struct's type.This also adds two helper classes,
StructLikeSetandPartitionSet. The struct set hidesStructLikeWrapperfrom callers and is based onCharSequenceSet.PartitionSetis a set implementation for partitions, which are identified by a partition spec id and a partition tuple.PartitionSetis needed because there are some cases where a partition tuple for different specs is identical. Updating existing classes to use these will be done in a follow-up PR.