Optimize partition value evaluation in filter pushdown

[feilong-liu]

Description

Constraint is used to validate whether a hive partition matches the filters specified in query during filter pushdown.

presto/presto-hive/src/main/java/com/facebook/presto/hive/HivePartitionManager.java

Line 417 in c08c6ef

if (constraint.predicate().isPresent() && !constraint.predicate().get().test(partition.getKeys())) {

Here constraint.predicate().get().test(partition.getKeys()) is called for one partition, and this function is called for all candidate partitions. However, it may not be necessary to call this function for all partitions.
For example, let's say we have a table with two partition keys, ds and ts, where ds is the date and ts is the hour of the day. When the query is querying data in the past 180 days, there can be 180*24=4320 candidate partitions. However, if the predicate in constraint only contains ds but not ts, we actually only need to call this function 180 times. It can lead to big difference especially when the predicate is expensive, for example large expressions or include complex string/json operations.
In the motivating example, we see query planning time decrease from 3 minutes to 10 seconds if we reduce the number of such function calls.
In this PR, I added a function to constraint to return the column handles which are used in predicate, and populate it during filter pushdown, so as to enable the above optimization.

Motivation and Context

Reduce query planning time.

Impact

Reduce query planning time.

Test Plan

Test with production queries end to end

Contributor checklist

• Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
• PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
• Documented new properties (with its default value), SQL syntax, functions, or other functionality.
• If release notes are required, they follow the release notes guidelines.
• Adequate tests were added if applicable.
• CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* Add a function to Constraint to return the input arguments for the predicate

[jaystarshot]

Where are the predicateInputs set and used?

[feilong-liu]

Where are the predicateInputs set and used?

I was thinking to split changes to spi change and the rest, updated the code to include the logic which uses this information too.

[feilong-liu]

Cache the result for predicates

[jaystarshot]

Can this cache blow up for very large distinct partition values ? maybe a guava cache is better?

[feilong-liu]

Thanks for the suggestion, just updated with guava cache

[jaystarshot]

LGTM , Is it easy to add a unit test in TestHivePartitionManager?

[feilong-liu]

LGTM , Is it easy to add a unit test in TestHivePartitionManager?

Sure, added a new test

[feilong-liu]

@jaystarshot Can you review it again? I just updated the TestHiveClientConfig to fix the failed test.

[steveburnett]

Thanks for the release note entry! Is there somewhere in the documentation that it could link to?