Skip to content

Fix bucket pruning when value is null and the predicate is 'IS NULL'.#14276

Merged
rongrong merged 1 commit intoprestodb:masterfrom
kaikalur:fix_null_bucket_pruning
Mar 24, 2020
Merged

Fix bucket pruning when value is null and the predicate is 'IS NULL'.#14276
rongrong merged 1 commit intoprestodb:masterfrom
kaikalur:fix_null_bucket_pruning

Conversation

@kaikalur
Copy link
Contributor

When filtering in buckets with matching values, handle NULL properly.

== RELEASE NOTES ==
Fixed bucket pruning when the predicate is IS NULL and the table actually has rows with that value which we were previously not considering.

@kaikalur kaikalur requested review from mbasmanova and rongrong March 22, 2020 17:14
@kaikalur kaikalur self-assigned this Mar 22, 2020
@kaikalur kaikalur force-pushed the fix_null_bucket_pruning branch from 53677ad to 8f17ad1 Compare March 22, 2020 21:01
@kaikalur kaikalur force-pushed the fix_null_bucket_pruning branch from 8f17ad1 to b40b151 Compare March 22, 2020 22:43
Copy link
Contributor

@mbasmanova mbasmanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaikalur The fix looks good, but the test should go into TestTupleDomain.

@mbasmanova mbasmanova requested a review from bhhari March 23, 2020 14:06
@mbasmanova mbasmanova assigned mbasmanova and unassigned kaikalur Mar 23, 2020
@mbasmanova mbasmanova added the bug label Mar 23, 2020
@kaikalur
Copy link
Contributor Author

kaikalur commented Mar 23, 2020

I disagree and I generally dislike unit tests. In fact, that function should not even be in there. They should have just called the other function that gives a map and extracted it. Too many ways to do the same thing is just bad.

@kaikalur
Copy link
Contributor Author

I disagree and I generally dislike unit tests. In fact, that function should not even be in there. They should have just called the other function that gives a map and extracted it. Too many ways to do the same thing is just bad.

Also, what if someone found a way to do this bucket pruning different way and missed the same thing. We will miss it again. This test is good like here.

@bhhari bhhari requested a review from oerling March 23, 2020 16:48
@bhhari
Copy link
Contributor

bhhari commented Mar 23, 2020

one more test case
assertQuery(session, "SELECT COUNT() FROM table_with_null_buckets WHERE key IN (NULL,1)", "SELECT 1");

@kaikalur
Copy link
Contributor Author

@kaikalur @mbasmanova the fix still fails for this case
assertQuery(session, "SELECT COUNT() FROM table_with_null_buckets WHERE key IN (NULL,1)", "SELECT 2");

It should be 1 because IN uses equality and key in (null,1) is same as key in (1). But good catch about missing test though!

@kaikalur kaikalur force-pushed the fix_null_bucket_pruning branch from b40b151 to 84ac3c6 Compare March 23, 2020 17:08
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nits: space after comma in NULL,1.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@rongrong
Copy link
Contributor

@kaikalur might be a good idea to also add unit test to TestTupleDomain in addition. Won't hurt.

@kaikalur
Copy link
Contributor Author

@kaikalur might be a good idea to also add unit test to TestTupleDomain in addition. Won't hurt.

It hurts me to do it :) and I'm also principally against unit testing.

If you ask me, this function does not belong here in this class.

@kaikalur
Copy link
Contributor Author

This is not good enough :(

        assertQuery(session, "SELECT COUNT() FROM table_with_null_buckets WHERE key IS NULL OR key = 1", "SELECT 2");

fails

@kaikalur
Copy link
Contributor Author

@mbasmanova can you check if this new fix works better? It fixed the failed test. I couldn't find an API that gives me a value for null so just adding null when it's nullAllowed seems to have fixed it.

@kaikalur kaikalur force-pushed the fix_null_bucket_pruning branch from 84ac3c6 to eff8c76 Compare March 23, 2020 20:25
Copy link
Contributor

@mbasmanova mbasmanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaikalur The new fix looks fine, but this kind of low-level changes really required a unit test. Without one it is not possible to fully understand whether all the possible scenarios are implemented correctly.

@kaikalur kaikalur force-pushed the fix_null_bucket_pruning branch from eff8c76 to 973b1b1 Compare March 23, 2020 22:11
@rongrong rongrong merged commit 6613a6a into prestodb:master Mar 24, 2020
@caithagoras caithagoras mentioned this pull request Mar 29, 2020
9 tasks
@caithagoras caithagoras mentioned this pull request May 4, 2020
34 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants