Intersect pushdown predicates

[oerling]

Predicates can be pushed down into a scan multiple times, for example
when a CTE has local predicates initially and acquires new predicates
from the enclosing context. Adds an intersect method to
DomainTranslater.ExtractionResult.

[highker]

minor comments

[highker]

Could you move this function inside HiveMetadata? Let's make the spi light-weighted.

[highker]

Put an empty line after this.

[highker]

Put an empty line after this.

[oerling]

OK. For future reference, you will likely want ExtractionResult.intersect In spi for symmetry with TupleDomain and because every other connector is likely to need this same thing. From: James Sun <notifications@github.com> Sent: Tuesday, October 8, 2019 7:35 PM To: prestodb/presto <presto@noreply.github.com> Cc: oerling <erling@xs4all.nl>; Author <author@noreply.github.com> Subject: Re: [prestodb/presto] Intersect pushdown predicates (#13505) @highker commented on this pull request. minor comments

_____ In presto-spi/src/main/java/com/facebook/presto/spi/relation/DomainTranslator.java <#13505 (comment)> :

@@ -55,6 +56,21 @@ public ExtractionResult(TupleDomain<T> tupleDomain, RowExpression remainingExpre

this.remainingExpression = requireNonNull(remainingExpression, "remainingExpression is null"); } + public ExtractionResult intersect(ExtractionResult other) Could you move this function inside HiveMetadata? Let's make the spi light-weighted.

_____ In presto-hive/src/main/java/com/facebook/presto/hive/HiveMetadata.java <#13505 (comment)> :

ExtractionResult<Subfield> decomposedFilter = rowExpressionService.getDomainTranslator()

.fromPredicate(session, filter, new SubfieldExtractor(functionResolution, rowExpressionService.getExpressionOptimizer(), session).toColumnExtractor()); - + if (currentLayoutHandle.isPresent()) { + HiveTableLayoutHandle currentHiveLayout = (HiveTableLayoutHandle) currentLayoutHandle.get(); + decomposedFilter = decomposedFilter.intersect(new ExtractionResult(currentHiveLayout.getDomainPredicate(), currentHiveLayout.getRemainingPredicate())); + } Put an empty line after this.

_____ In presto-hive/src/main/java/com/facebook/presto/hive/HiveMetadata.java <#13505 (comment)> :

Map<String, ColumnHandle> columnHandles = getColumnHandles(session, tableHandle);

TupleDomain<ColumnHandle> entireColumnDomain = decomposedFilter.getTupleDomain() .transform(subfield -> isEntireColumn(subfield) ? subfield.getRootName() : null) .transform(columnHandles::get); - // TODO Extract deterministic conjuncts that apply to partition columns and specify these as Contraint#predicate + if (currentLayoutHandle.isPresent()) { + entireColumnDomain = entireColumnDomain.intersect(((HiveTableLayoutHandle) (currentLayoutHandle.get())).getPartitionColumnPredicate()); + } Put an empty line after this. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#13505?email_source=notifications&email_token=AKPPPT6MFYE3JIKQELWCYOTQNU7MRA5CNFSM4I6ABKJKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCHKI2PQ#pullrequestreview-299142462> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AKPPPT6RRHKQ722GGOMLZZDQNU7MRANCNFSM4I6ABKJA> .

[bhhari]

lgtm

[rschlussel]

looks good, thanks.

[oerling]

Could you or James merge this? Thanks Orri From: Rebecca Schlussel <notifications@github.com> Sent: Friday, October 11, 2019 8:06 AM To: prestodb/presto <presto@noreply.github.com> Cc: oerling <erling@xs4all.nl>; Author <author@noreply.github.com> Subject: Re: [prestodb/presto] Intersect pushdown predicates (#13505) @rschlussel approved this pull request. looks good, thanks. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#13505?email_source=notifications&email_token=AKPPPT4H6FDNV6T4ULIWB5LQOCI5RA5CNFSM4I6ABKJKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCHWQ4FQ#pullrequestreview-300748310> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKPPPTZN7T4HCCVBV2KXMNLQOCI5RANCNFSM4I6ABKJA> .

[rschlussel]

@highker did you have any additional comments?

[highker]

LGTM

[highker]

@oerling, could you rebase so we could merge?

[oerling]

OK, rebased. From: James Sun <notifications@github.com> Sent: Wednesday, October 16, 2019 10:35 AM To: prestodb/presto <presto@noreply.github.com> Cc: oerling <erling@xs4all.nl>; Mention <mention@noreply.github.com> Subject: Re: [prestodb/presto] Intersect pushdown predicates (#13505) @oerling <https://github.com/oerling> , could you rebase so we could merge? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13505?email_source=notifications&email_token=AKPPPT5ITWXDOPWN7YPSRMTQO5GDDA5CNFSM4I6ABKJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBNJ7MY#issuecomment-542810035> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKPPPT4KND2LHH3E23HCL7LQO5GDDANCNFSM4I6ABKJA> .