Skip to content

Update HiveTableHandle equals#1448

Merged
findepi merged 1 commit intotrinodb:masterfrom
findepi:findepi/master/update-hivetablehandle-equals-c0c9bd
Nov 8, 2019
Merged

Update HiveTableHandle equals#1448
findepi merged 1 commit intotrinodb:masterfrom
findepi:findepi/master/update-hivetablehandle-equals-c0c9bd

Conversation

@findepi
Copy link
Copy Markdown
Member

@findepi findepi commented Sep 5, 2019

HiveTableHandle.equals() got simplified in
a389636.

@cla-bot cla-bot bot added the cla-signed label Sep 5, 2019
@findepi findepi requested a review from electrum September 14, 2019 19:49
@findepi findepi requested review from electrum and kokosing and removed request for electrum September 25, 2019 09:40
public int hashCode()
{
return Objects.hash(schemaName, tableName);
return Objects.hash(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need hashCode/equals at all for ConnectorTableHandle? If so then then they should be listed in SPI. How did you find that? Is there any Presto behavior change with this? Maybe we should remove these methods from Presto if they are not used?

Objects.equals(tableParameters, that.tableParameters) &&
Objects.equals(partitionColumns, that.partitionColumns) &&
Objects.equals(partitions, that.partitions) &&
Objects.equals(compactEffectivePredicate, that.compactEffectivePredicate) &&
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of these fields (e.g., compactEffectivePredicate, enforcedPredicate) are not part of the identity of a hive "virtual" table. Handles should be equal to each other if they represent the same "object" (handles should be thought of like opaque IDs).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a handle represents table+predicate, it's no longer "the same" as handle representing a table.
Later on we will likely introduce table handles modelling a pushed down join.

Maybe we should remove equals from table handles? And treat them truly opaque?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@martint @electrum your feedback would be appreciated.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should remove equals from table handles? And treat them truly opaque?

They are opaque (from the point of view that the engine doesn't care about what's inside), but just like an id, you should be able to test id1 == id2 to see if they are the same. At some point we'll need that for some forms of memoization in the optimizer.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, so they are opaque but need to implement equals().

  • we need document this requirement at SPI interface level
  • HiveTableHandle as it is today doesn't meet memo's requirements, since it ignores part of its (important) state in equals

if you agree, we should proceed with the PR, right?

@findepi findepi force-pushed the findepi/master/update-hivetablehandle-equals-c0c9bd branch from 6c8c1fc to 894953b Compare October 30, 2019 22:24
@findepi
Copy link
Copy Markdown
Member Author

findepi commented Oct 30, 2019

Rebased to re-run travis. (The change was last tested against 318)

@findepi findepi requested a review from martint November 2, 2019 11:57
@findepi findepi force-pushed the findepi/master/update-hivetablehandle-equals-c0c9bd branch from 894953b to cdc87e8 Compare November 8, 2019 09:53
`HiveTableHandle.equals()` got simplified in
a389636.
@findepi findepi force-pushed the findepi/master/update-hivetablehandle-equals-c0c9bd branch from cdc87e8 to 49e6255 Compare November 8, 2019 12:41
@findepi findepi merged commit 9500f12 into trinodb:master Nov 8, 2019
@findepi findepi deleted the findepi/master/update-hivetablehandle-equals-c0c9bd branch November 8, 2019 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants