Skip to content

Fix incorrect $bucket for mismatch bucket queries#11885

Merged
arhimondr merged 1 commit intotrinodb:masterfrom
oraclechang:support_mismatch_bucket_query2
May 7, 2022
Merged

Fix incorrect $bucket for mismatch bucket queries#11885
arhimondr merged 1 commit intotrinodb:masterfrom
oraclechang:support_mismatch_bucket_query2

Conversation

@oraclechang
Copy link
Copy Markdown
Member

@oraclechang oraclechang commented Apr 8, 2022

Comparable change: prestodb/presto#12429

This change allows executing queries with the following conditions. Currently, these queries will raise an exception.

if (tableBucketCount != readBucketCount && bucketFilter.isPresent())

A more specific example can be seen in the testMismatchedBucketWithBucketPredicate() test.

@cla-bot cla-bot bot added the cla-signed label Apr 8, 2022
@findepi findepi requested a review from electrum April 9, 2022 08:33
@oraclechang oraclechang force-pushed the support_mismatch_bucket_query2 branch from 3580a93 to f78d3a1 Compare April 12, 2022 00:30
findepi
findepi reviewed Apr 13, 2022
View reviewed changes
plugin/trino-hive/src/test/java/io/trino/plugin/hive/BaseHiveConnectorTest.java Outdated
Copy link
Copy Markdown
Member

@findepi findepi Apr 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this testable with smaller tables?
Like nation with bucketing on nationkey?

Copy link
Copy Markdown
Member Author

@oraclechang oraclechang Apr 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

plugin/trino-hive/src/test/java/io/trino/plugin/hive/BaseHiveConnectorTest.java Outdated
Copy link
Copy Markdown
Member

@findepi findepi Apr 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the try-finally isn't really needed in this class, since all the test data is ephemeral anyway

Copy link
Copy Markdown
Member Author

@oraclechang oraclechang Apr 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the try-finally

plugin/trino-hive/src/test/java/io/trino/plugin/hive/BaseHiveConnectorTest.java Outdated
Copy link
Copy Markdown
Member

@findepi findepi Apr 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it matter? document why

Copy link
Copy Markdown
Member Author

@oraclechang oraclechang Apr 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

plugin/trino-hive/src/test/java/io/trino/plugin/hive/BaseHiveConnectorTest.java Outdated
Copy link
Copy Markdown
Member

@findepi findepi Apr 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: \n are redundant, replace with spaces so that it's more readable

Copy link
Copy Markdown
Member Author

@oraclechang oraclechang Apr 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

@oraclechang oraclechang force-pushed the support_mismatch_bucket_query2 branch 2 times, most recently from 3ce8645 to 185deca Compare April 15, 2022 00:39
@oraclechang oraclechang requested a review from findepi April 18, 2022 17:14
@oraclechang
Copy link
Copy Markdown
Member Author

oraclechang commented Apr 25, 2022 •
edited
Loading

Hi @findepi, @electrum

may I have a review please?
Thanks!

@oraclechang oraclechang requested a review from arhimondr May 3, 2022 21:06
@oraclechang
Copy link
Copy Markdown
Member Author

oraclechang commented May 3, 2022

Hi @arhimondr,

Can you please have a look?
Thanks!

arhimondr
arhimondr approved these changes May 6, 2022
View reviewed changes
plugin/trino-hive/src/main/java/io/trino/plugin/hive/HiveSplitManager.java Outdated
Copy link
Copy Markdown
Contributor

@arhimondr arhimondr May 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please open a follow up PR dropping the HivePartitioningHandle#maxCompatibleBucketCount. Currently it is effectively unused as number of read buckets higher than number of table buckets is no longer supported.

Copy link
Copy Markdown
Member Author

@oraclechang oraclechang May 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

@arhimondr
Copy link
Copy Markdown
Contributor

arhimondr commented May 6, 2022

@oraclechang Could you please rebase and resubmit to make sure it compiles on Trunk. I will merge once the build is green.

@oraclechang oraclechang force-pushed the support_mismatch_bucket_query2 branch from 185deca to 9adf088 Compare May 6, 2022 23:36
@arhimondr arhimondr merged commit 4dd11db into trinodb:master May 7, 2022
@github-actions github-actions bot added this to the 381 milestone May 7, 2022
@mosabua mosabua mentioned this pull request May 9, 2022
Merged
@oraclechang oraclechang deleted the support_mismatch_bucket_query2 branch May 12, 2022 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@electrum electrum Awaiting requested review from electrum

@findepi findepi Awaiting requested review from findepi

1 more reviewer

@arhimondr arhimondr arhimondr approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

cla-signed

Milestone

381

Development

Successfully merging this pull request may close these issues.

3 participants

@oraclechang @arhimondr @findepi