Enable logical property propagation by default

[vivek-bharathan]

Fixes #21968

== RELEASE NOTES ==

General Changes
* Enable propagation of logical properties by default

[steveburnett]

The only place in the docs that exploit_constraints or optimizer.exploit-constraints are mentioned is in the 0.274 release notes. Could docs for these two properties be added as part of this PR?

[github-actions]

Codenotify: Notifying subscribers in CODENOTIFY files for diff 9dfc461...de6f59e.

Notify	File(s)
@steveburnett	presto-docs/src/main/sphinx/admin/properties.rst presto-docs/src/main/sphinx/optimizer.rst presto-docs/src/main/sphinx/optimizer/logical-properties.rst

[kaikalur]

I'm not sure we want to do this by default (unless the optimizations that use the property are off by default). We are not sure how they would affect our prod workloads. In any case, an installation can always turn it on cluster-wide using feature config. So I say keep it off for now.

[steveburnett]

Thanks! A few nits on formatting and punctuation.

You must add optimizer/logical-properties to https://github.com/prestodb/presto/blob/master/presto-docs/src/main/sphinx/optimizer.rst for the new page to be part of the left navbar.

Consider adding entries for exploit_constraints and optimizer.exploit-constraints to the Properties Reference doc, maybe in the Optimizer Properties topic.

[tdcmeehan]

I'm not sure we want to do this by default (unless the optimizations that use the property are off by default). We are not sure how they would affect our prod workloads. In any case, an installation can always turn it on cluster-wide using feature config. So I say keep it off for now.

If you're worried about that, can't you just disable it for your deployment and figure out a way (if any) to enable it gradually?

[kaikalur]

I'm not sure we want to do this by default (unless the optimizations that use the property are off by default). We are not sure how they would affect our prod workloads. In any case, an installation can always turn it on cluster-wide using feature config. So I say keep it off for now.

If you're worried about that, can't you just disable it for your deployment and figure out a way (if any) to enable it gradually?

Yeah - we were just talking about it. We will shut it off on our side.

[ZacBlanco]

minor nit, otherwise lgtm

[steveburnett]

Looks good! A couple of nits, nothing big.

[steveburnett]

LGTM! (docs)

Pull updated branch, new local build.

[feilong-liu]

@ClarenceThreepwood @tdcmeehan

We found a correctness bug for the logical property propagation framework, a reproduce query is as follows:

Returns 0 rows when the framework is on:

presto:tpch> set session exploit_constraints=true;
SET SESSION
presto:tpch> with t as (select orderkey, count(1) cnt from (select * from (select * from orders where 1=0) left join (select partkey, suppkey from lineitem where 1=0) on partkey=10 where suppkey is not null) group by rollup(orderkey)) select t1.orderkey, t1.cnt from t t1 cross join t t2;
 orderkey | cnt 
----------+-----
(0 rows)

Returns 1 row when the framework is off:

presto:tpch> set session exploit_constraints=false;
SET SESSION
presto:tpch> with t as (select orderkey, count(1) cnt from (select * from (select * from orders where 1=0) left join (select partkey, suppkey from lineitem where 1=0) on partkey=10 where suppkey is not null) group by rollup(orderkey)) select t1.orderkey, t1.cnt from t t1 cross join t t2;
 orderkey | cnt 
----------+-----
 NULL     |   0 
(1 row)

Looking into the query plan, looks like that the framework incorrectly oversimplify the plan
When framework is on:

presto:tpch> set session exploit_constraints=true;
SET SESSION
presto:tpch> explain (type distributed) with t as (select orderkey, count(1) cnt from (select * from (select * from orders where 1=0) left join (select partkey, suppkey from lineitem where 1=0) on partkey=10 where suppkey is not null) group by rollup(orderkey)) select t1.orderkey, t1.cnt from t t1 cross join t t2;
                                                                                                   Query Plan                                                                                                   
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Fragment 0 [SINGLE]                                                                                                                                                                                            
     Output layout: [orderkey$gid, count]                                                                                                                                                                       
     Output partitioning: SINGLE []                                                                                                                                                                             
     Stage Execution Strategy: UNGROUPED_EXECUTION                                                                                                                                                              
     - Output[PlanNodeId 63][orderkey, cnt] => [orderkey$gid:bigint, count:bigint]                                                                                                                              
             orderkey := orderkey$gid (1:257)                                                                                                                                                                   
             cnt := count (1:270)                                                                                                                                                                               
         - CrossJoin[PlanNodeId 370] => [orderkey$gid:bigint, count:bigint]                                                                                                                                     
                 Distribution: REPLICATED                                                                                                                                                                       
             - Project[PlanNodeId 24][projectLocality = LOCAL] => [orderkey$gid:bigint, count:bigint]                                                                                                           
                 - Aggregate(FINAL)[orderkey$gid, groupid][$hashvalue][PlanNodeId 23] => [orderkey$gid:bigint, groupid:bigint, $hashvalue:bigint, count:bigint]                                                 
                         count := "presto.default.count"((count_246)) (1:56)                                                                                                                                    
                     - LocalExchange[PlanNodeId 1597][HASH][$hashvalue] (orderkey$gid, groupid) => [orderkey$gid:bigint, groupid:bigint, count_246:bigint, $hashvalue:bigint]                                   
                         - Aggregate(PARTIAL)[orderkey$gid, groupid][$hashvalue_247][PlanNodeId 1595] => [orderkey$gid:bigint, groupid:bigint, $hashvalue_247:bigint, count_246:bigint]                         
                                 count_246 := "presto.default.count"((expr_93)) (1:56)                                                                                                                          
                             - Project[PlanNodeId 1624][projectLocality = LOCAL] => [orderkey$gid:bigint, expr_93:integer, groupid:bigint, $hashvalue_247:bigint]                                               
                                     Estimates: {source: CostBasedSourceInfo, rows: 0 (0B), cpu: 0.00, memory: 0.00, network: 0.00}                                                                             
                                     $hashvalue_247 := combine_hash(combine_hash(BIGINT'0', COALESCE($operator$hash_code(orderkey$gid), BIGINT'0')), COALESCE($operator$hash_code(groupid), BIGINT'0')) (1:239) 
                                 - Values[PlanNodeId 1474] => [orderkey$gid:bigint, expr_93:integer, groupid:bigint]                                                                                            
                                         Estimates: {source: CostBasedSourceInfo, rows: 0 (0B), cpu: 0.00, memory: 0.00, network: 0.00}                                                                         
             - Values[PlanNodeId 1478] => []                                                                                                                                                                    
                     Estimates: {source: CostBasedSourceInfo, rows: 0 (0B), cpu: 0.00, memory: 0.00, network: 0.00}                                                                                             
                                                                                                                                                                                                                
                                                                                                                                                                                                                
(1 row)

When framework is off:

presto:tpch> set session exploit_constraints=false;
SET SESSION
presto:tpch> explain (type distributed) with t as (select orderkey, count(1) cnt from (select * from (select * from orders where 1=0) left join (select partkey, suppkey from lineitem where 1=0) on partkey=10 where suppkey is not null) group by rollup(orderkey)) select t1.orderkey, t1.cnt from t t1 cross join t t2;
                                                                                                         Query Plan                                                                                                >
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
 Fragment 0 [SINGLE]                                                                                                                                                                                               >
     Output layout: [orderkey$gid, count]                                                                                                                                                                          >
     Output partitioning: SINGLE []                                                                                                                                                                                >
     Stage Execution Strategy: UNGROUPED_EXECUTION                                                                                                                                                                 >
     - Output[PlanNodeId 63][orderkey, cnt] => [orderkey$gid:bigint, count:bigint]                                                                                                                                 >
             orderkey := orderkey$gid (1:257)                                                                                                                                                                      >
             cnt := count (1:270)                                                                                                                                                                                  >
         - CrossJoin[PlanNodeId 370] => [orderkey$gid:bigint, count:bigint]                                                                                                                                        >
                 Distribution: REPLICATED                                                                                                                                                                          >
             - Project[PlanNodeId 24][projectLocality = LOCAL] => [orderkey$gid:bigint, count:bigint]                                                                                                              >
                 - Aggregate(FINAL)[orderkey$gid, groupid][$hashvalue][PlanNodeId 23] => [orderkey$gid:bigint, groupid:bigint, $hashvalue:bigint, count:bigint]                                                    >
                         count := "presto.default.count"((count_246)) (1:56)                                                                                                                                       >
                     - LocalExchange[PlanNodeId 1619][HASH][$hashvalue] (orderkey$gid, groupid) => [orderkey$gid:bigint, groupid:bigint, count_246:bigint, $hashvalue:bigint]                                      >
                         - Aggregate(PARTIAL)[orderkey$gid, groupid][$hashvalue_247][PlanNodeId 1617] => [orderkey$gid:bigint, groupid:bigint, $hashvalue_247:bigint, count_246:bigint]                            >
                                 count_246 := "presto.default.count"((expr_93)) (1:56)                                                                                                                             >
                             - Project[PlanNodeId 1673][projectLocality = LOCAL] => [orderkey$gid:bigint, expr_93:integer, groupid:bigint, $hashvalue_247:bigint]                                                  >
                                     Estimates: {source: CostBasedSourceInfo, rows: 0 (0B), cpu: 0.00, memory: 0.00, network: 0.00}                                                                                >
                                     $hashvalue_247 := combine_hash(combine_hash(BIGINT'0', COALESCE($operator$hash_code(orderkey$gid), BIGINT'0')), COALESCE($operator$hash_code(groupid), BIGINT'0')) (1:239)    >
                                 - Values[PlanNodeId 1474] => [orderkey$gid:bigint, expr_93:integer, groupid:bigint]                                                                                               >
                                         Estimates: {source: CostBasedSourceInfo, rows: 0 (0B), cpu: 0.00, memory: 0.00, network: 0.00}                                                                            >
             - LocalExchange[PlanNodeId 1602][SINGLE] () => []                                                                                                                                                     >
                 - Project[PlanNodeId 53][projectLocality = LOCAL] => []                                                                                                                                           >
                     - Aggregate(FINAL)[orderkey$gid_225, groupid_226][$hashvalue_248][PlanNodeId 52] => [orderkey$gid_225:bigint, groupid_226:bigint, $hashvalue_248:bigint]                                      >
                         - LocalExchange[PlanNodeId 1629][HASH][$hashvalue_248] (orderkey$gid_225, groupid_226) => [orderkey$gid_225:bigint, groupid_226:bigint, $hashvalue_248:bigint]                            >
                             - Aggregate(PARTIAL)[orderkey$gid_225, groupid_226][$hashvalue_249][PlanNodeId 1627] => [orderkey$gid_225:bigint, groupid_226:bigint, $hashvalue_249:bigint]                          >
                                 - Project[PlanNodeId 1674][projectLocality = LOCAL] => [orderkey$gid_225:bigint, groupid_226:bigint, $hashvalue_249:bigint]                                                       >
                                         Estimates: {source: CostBasedSourceInfo, rows: 0 (0B), cpu: 0.00, memory: 0.00, network: 0.00}                                                                            >
                                         $hashvalue_249 := combine_hash(combine_hash(BIGINT'0', COALESCE($operator$hash_code(orderkey$gid_225), BIGINT'0')), COALESCE($operator$hash_code(groupid_226), BIGINT'0'))>
                                     - Values[PlanNodeId 1478] => [orderkey$gid_225:bigint, groupid_226:bigint]                                                                                                    >
                                             Estimates: {source: CostBasedSourceInfo, rows: 0 (0B), cpu: 0.00, memory: 0.00, network: 0.00}                                                                        >
                                                                                                                                                                                                                   >
                                                                                                                                                                                                                   >
(1 row)