Skip to content

Track applicable optimizations that were disabled with a flag#19772

Merged
mlyublena merged 1 commit intoprestodb:masterfrom
mlyublena:track-applicable-optimizer-rules
Jun 13, 2023
Merged

Track applicable optimizations that were disabled with a flag#19772
mlyublena merged 1 commit intoprestodb:masterfrom
mlyublena:track-applicable-optimizer-rules

Conversation

@mlyublena
Copy link
Contributor

@mlyublena mlyublena commented Jun 1, 2023

This PR extends previous change, which tracked which rule-based optimizations triggered for a query in the following shape:

RemoveRedundantIdentityProjections(triggered: true, applicable: true)
PruneTableScanColumns(triggered: true, applicable: true)
...

This PR adds the following:

  • also track plan optimizers (and not just rule-based optimizers)
  • track which optimizers were applicable but did not trigger because they are disabled by a flag:
PayloadJoinOptimizer(triggered: false, applicable: true)
  • adds info about triggered and applicable optimizers to EXPLAIN output, e.g.:
presto:tpch> explain SELECT l.* FROM lineitem_map l left join orders o on (l.orderkey+1 = o.orderkey+1) left join part p on (l.partkey+1=p.partkey+1);
                                                                                                                                                                                 >
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
 - Output[orderkey, partkey, suppkey, linenumber, quantity, extendedprice, discount, tax, returnflag, linestatus, shipdate, commitdate, receiptdate, shipinstruct, shipmode, comm>
         Estimates: {rows: 135354 (24.12MB), cpu: 176007121.33, memory: 306000.00, network: 56242251.93}                                                                         >
     - RemoteStreamingExchange[GATHER] => [orderkey:bigint, partkey:bigint, suppkey:bigint, linenumber:integer, quantity:double, extendedprice:double, discount:double, tax:doubl>
             Estimates: {rows: 135354 (24.12MB), cpu: 176007121.33, memory: 306000.00, network: 56242251.93}                                                                     >
         - LeftJoin[("expr_11" = "expr_12")][$hashvalue, $hashvalue_93] => [orderkey:bigint, partkey:bigint, suppkey:bigint, linenumber:integer, quantity:double, extendedprice:d>
....                                                                                                >
 Triggered optimizers: [RemoveRedundantIdentityProjections, PruneTableScanColumns, PruneJoinColumns, PruneProjectColumns, PruneJoinChildrenColumns, ProjectRowExpressionRewrite, >
 Applicable optimizers: [AddNotNullFiltersToJoinNode, PayloadJoinOptimizer]

This information is only printed if the flag verbose_optimizer_info_enabled is true.

@mlyublena mlyublena requested a review from a team as a code owner June 1, 2023 16:52
@mlyublena mlyublena requested a review from ajaygeorge June 1, 2023 16:52
@mlyublena mlyublena marked this pull request as draft June 1, 2023 16:53
@mlyublena mlyublena requested a review from feilong-liu June 1, 2023 16:53
@mlyublena mlyublena force-pushed the track-applicable-optimizer-rules branch 4 times, most recently from b9bb478 to d96fe63 Compare June 5, 2023 22:10
@mlyublena mlyublena marked this pull request as ready for review June 5, 2023 22:21
@mlyublena mlyublena requested a review from pranjalssh June 5, 2023 22:21
@mlyublena mlyublena force-pushed the track-applicable-optimizer-rules branch from d96fe63 to a67f5ba Compare June 6, 2023 23:56
@jainxrohit
Copy link
Contributor

Failing with check style

@mlyublena mlyublena force-pushed the track-applicable-optimizer-rules branch 2 times, most recently from 608d2c4 to 945bc50 Compare June 8, 2023 22:10
@mlyublena mlyublena force-pushed the track-applicable-optimizer-rules branch from e3b484b to 2a72c33 Compare June 12, 2023 23:35
@mlyublena mlyublena force-pushed the track-applicable-optimizer-rules branch from 2a72c33 to c25c10c Compare June 13, 2023 05:14
@mlyublena mlyublena merged commit 5039ce5 into prestodb:master Jun 13, 2023
Copy link
Contributor

@mbasmanova mbasmanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mlyublena PrefilterForLimitingAggregation.java.orig seems to be blocking diff train. Why is this file part of the PR? Looks like a left over from debugging / development.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants