Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion datafusion/physical-plan/src/aggregates/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1147,7 +1147,7 @@ impl AggregateExec {
} else if fun_name.eq_ignore_ascii_case("max") {
DynamicFilterAggregateType::Max
} else {
continue;
return;
};

// 2. arg should be only 1 column reference
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,42 @@ physical_plan
04)------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
05)--------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/dynamic_filter_pushdown_config/agg_data.parquet]]}, projection=[score], file_type=parquet, predicate=category@0 = alpha AND DynamicFilter [ empty ], pruning_predicate=category_null_count@2 != row_count@3 AND category_min@0 <= alpha AND alpha <= category_max@1, required_guarantees=[category in (alpha)]

# Test 4b: Mixed aggregates with non MIN/MAX should not have DynamicFilter

# Pure MAX — DynamicFilter is expected here as we can safely prune that don't match the filter.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is already a test for this

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed that one! thanks!

query TT
EXPLAIN SELECT MAX(score) FROM agg_parquet WHERE category = 'alpha';
----
logical_plan
01)Aggregate: groupBy=[[]], aggr=[[max(agg_parquet.score)]]
02)--Projection: agg_parquet.score
03)----Filter: agg_parquet.category = Utf8View("alpha")
04)------TableScan: agg_parquet projection=[category, score], partial_filters=[agg_parquet.category = Utf8View("alpha")]
physical_plan
01)AggregateExec: mode=Final, gby=[], aggr=[max(agg_parquet.score)]
02)--CoalescePartitionsExec
03)----AggregateExec: mode=Partial, gby=[], aggr=[max(agg_parquet.score)]
04)------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
05)--------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/dynamic_filter_pushdown_config/agg_data.parquet]]}, projection=[score], file_type=parquet, predicate=category@0 = alpha AND DynamicFilter [ empty ], pruning_predicate=category_null_count@2 != row_count@3 AND category_min@0 <= alpha AND alpha <= category_max@1, required_guarantees=[category in (alpha)]

# COUNT + MAX — DynamicFilter should NOT appear here in mixed aggregates
query TT
EXPLAIN SELECT COUNT(*), MAX(score) FROM agg_parquet WHERE category = 'alpha';
----
logical_plan
01)Projection: count(Int64(1)) AS count(*), max(agg_parquet.score)
02)--Aggregate: groupBy=[[]], aggr=[[count(Int64(1)), max(agg_parquet.score)]]
03)----Projection: agg_parquet.score
04)------Filter: agg_parquet.category = Utf8View("alpha")
05)--------TableScan: agg_parquet projection=[category, score], partial_filters=[agg_parquet.category = Utf8View("alpha")]
physical_plan
01)ProjectionExec: expr=[count(Int64(1))@0 as count(*), max(agg_parquet.score)@1 as max(agg_parquet.score)]
02)--AggregateExec: mode=Final, gby=[], aggr=[count(Int64(1)), max(agg_parquet.score)]
03)----CoalescePartitionsExec
04)------AggregateExec: mode=Partial, gby=[], aggr=[count(Int64(1)), max(agg_parquet.score)]
05)--------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
06)----------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/dynamic_filter_pushdown_config/agg_data.parquet]]}, projection=[score], file_type=parquet, predicate=category@0 = alpha, pruning_predicate=category_null_count@2 != row_count@3 AND category_min@0 <= alpha AND alpha <= category_max@1, required_guarantees=[category in (alpha)]

# Disable aggregate dynamic filters only
statement ok
SET datafusion.optimizer.enable_aggregate_dynamic_filter_pushdown = false;
Expand Down
Loading