[native] Push down FilterNode into TableScan#23755
[native] Push down FilterNode into TableScan#23755aditi-pandit merged 1 commit intoprestodb:masterfrom
Conversation
493b3ce to
8c97879
Compare
There was a problem hiding this comment.
Thanks @Yuhta. Had a high level question:
Am curious why we are doing the pushdown of FilterNode into TableScan during Presto -> Velox plan conversion ? This could be an optimization in Velox Local Planner as well. That would make it common to Presto and Spark.
presto-native-execution/presto_cpp/main/types/PrestoToVeloxConnector.cpp
Show resolved
Hide resolved
|
@aditi-pandit Local planner has no dependency on Hive so it does not have knowledge about remaining filter. I guess this was the same reason why Presto planner does not put it in remaining filter as well (did not verify). Seems Prestissimo is a reasonable place to fix this, as we have all the dependency needed and we have the knowledge that this will benefit the execution layer (Velox). |
presto-native-execution/presto_cpp/main/types/PrestoToVeloxQueryPlan.cpp
Show resolved
Hide resolved
Hmm... Yeah, Velox Local Planner doesn't have dependency on Hive. Alright, your approach is reasonable. |
8c97879 to
ee4bb8e
Compare
Also add `$path` and `$bucket` in split info columns and fix split counts in coordinator UI.
ee4bb8e to
cff51f6
Compare
aditi-pandit
left a comment
There was a problem hiding this comment.
Thanks @Yuhta for this code, and answering all my questions.
|
Please consider adding release notes following our release notes guide - link. Using below for now
|
Sometimes the planner put a
FilterNoderight afterTableScanNode. This change merges these 2 nodes by moving the filter expression into remaining filter of the table scan when possible. This results in fewer IO and CPU because table scan can leverage the information in remaining filter to skip file stripes in case of random sampling.Also add
$pathand$bucketin split info columns and fix split counts in coordinator UI.