[SQL][Minor] Refactors deeply nested FP style code in BooleanSimplification #4091

liancheng · 2015-01-18T01:58:56Z

This is a follow-up of #4090. The original deeply nested reduceOption code is hard to grasp.

SparkQA · 2015-01-18T02:02:43Z

Test build #25712 has started for PR 4091 at commit e833ca4.

This patch merges cleanly.

SparkQA · 2015-01-18T03:11:49Z

Test build #25712 has finished for PR 4091 at commit e833ca4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-01-18T03:11:52Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25712/
Test PASSed.

SparkQA · 2015-01-18T07:47:40Z

Test build #25717 has started for PR 4091 at commit 1bf3258.

This patch merges cleanly.

SparkQA · 2015-01-18T08:32:42Z

Test build #25718 has started for PR 4091 at commit cd8860b.

This patch merges cleanly.

SparkQA · 2015-01-18T08:55:28Z

Test build #25717 has finished for PR 4091 at commit 1bf3258.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-01-18T08:55:31Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25717/
Test PASSed.

SparkQA · 2015-01-18T09:40:03Z

Test build #25718 has finished for PR 4091 at commit cd8860b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-01-18T09:40:06Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25718/
Test PASSed.

liancheng · 2015-01-20T07:15:37Z

Hey @scwf, would you mind to take a look at this? Thanks!

scwf · 2015-01-20T12:04:08Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala

-          if (ldiff.size == 0 || rdiff.size == 0) {
-            // a && (a || b) => a
-            common.reduce(Or)
+          if (common.isEmpty) {


we can remove this if

Yeah, the logic here is redundant. This if is suggested by @rxin as a shortcut for the case no common factors can be found to save query planning time.

liancheng · 2015-01-20T18:54:05Z

@rxin This should be ready to go.

rxin · 2015-01-20T19:20:06Z

Merging in master. Thanks.

…cation This is a follow-up of apache#4090. The original deeply nested `reduceOption` code is hard to grasp.  [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/apache/spark/4091)  Author: Cheng Lian <[email protected]> Closes apache#4091 from liancheng/refactor-boolean-simplification and squashes the following commits: cd8860b [Cheng Lian] Improves `compareConditions` to handle more subtle cases 1bf3258 [Cheng Lian] Avoids converting predicate sets to lists e833ca4 [Cheng Lian] Refactors deeply nested FP style code

…42.7.4 and `mssql` to 12.8.1.jre11 ### What changes were proposed in this pull request? This PR aims to upgrade `h2` to 2.3.232, `postgresql` to 42.7.4 and `mssql` to 12.8.1.jre11. ### Why are the changes needed? 1. For `h2`, there are some issues fixed in version 2.3.232(full release notes: https://www.h2database.com/html/changelog.html): - [Issue #3945](h2database/h2database#3945): Column not found in correlated subquery, when referencing outer column from LEFT JOIN .. ON clause - [Issue #4097](h2database/h2database#4097): StackOverflowException when using multiple SELECT statements in one query (2.3.230) - [Issue #3982](h2database/h2database#3982): Potential issue when using ROUND - [Issue #3894](h2database/h2database#3894): Race condition causing stale data in query last result cache - [Issue #4075](h2database/h2database#4075): infinite loop in compact - [Issue #4091](h2database/h2database#4091): Wrong case with linked table to postgresql - [Issue #4088](h2database/h2database#4088): BadGrammarException when the same alias is used within two different CTEs 2. For `postgresql`, there are some issues fixed and improvements in version 42.7.4(full release notes: https://jdbc.postgresql.org/changelogs/2024-08-22-42.7.4-release/): - fix: PgInterval ignores case for represented interval string [PR #3344](pgjdbc/pgjdbc#3344) - perf: Avoid extra copies when receiving int4 and int2 in PGStream [PR #3295](pgjdbc/pgjdbc#3295) - fix: Add support for Infinity::numeric values in ResultSet.getObject [PR #3304](pgjdbc/pgjdbc#3304) - fix: Ensure order of results for getDouble [PR #3301](pgjdbc/pgjdbc#3301) - perf: Replace BufferedOutputStream with unsynchronized PgBufferedOutputStream, allow configuring different Java and SO_SNDBUF buffer sizes [PR #3248](pgjdbc/pgjdbc#3248) - fix: Fix SSL tests [PR #3260](pgjdbc/pgjdbc#3260) - fix: Support bytea in preferQueryMode=simple [PR #3243](pgjdbc/pgjdbc#3243) - fix: Fix [Issue #3234](pgjdbc/pgjdbc#3234) - Return -1 as update count for stored procedure calls [PR #3235](pgjdbc/pgjdbc#3235) - fix: Fix [Issue #3224](pgjdbc/pgjdbc#3224) - conversion for TIME ‘24:00’ to LocalTime breaks in binary-mode [PR #3225](pgjdbc/pgjdbc#3225) 3. For `mssql`, there are some issues fixed in 12.8.1.jre11(full release notes: https://github.com/microsoft/mssql-jdbc/releases/tag/v12.8.1): - Adjusted DESTINATION_COL_METADATA_LOCK, in SQLServerBulkCopy, so that is properly released in all cases [PR #2492](microsoft/mssql-jdbc#2492) - Reverted "Execute Stored Procedures Directly" feature, as well as subsequent changes related to the feature [PR #2493](microsoft/mssql-jdbc#2493) - Changed driver behavior to allow prepared statement objects to be reused, preventing a "multiple queries are not allowed" error [PR #2494](microsoft/mssql-jdbc#2494) ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass GA. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #47810 from wayneguow/ug_h2. Authored-by: Wei Guo <[email protected]> Signed-off-by: Kent Yao <[email protected]>

…42.7.4 and `mssql` to 12.8.1.jre11 ### What changes were proposed in this pull request? This PR aims to upgrade `h2` to 2.3.232, `postgresql` to 42.7.4 and `mssql` to 12.8.1.jre11. ### Why are the changes needed? 1. For `h2`, there are some issues fixed in version 2.3.232(full release notes: https://www.h2database.com/html/changelog.html): - [Issue apache#3945](h2database/h2database#3945): Column not found in correlated subquery, when referencing outer column from LEFT JOIN .. ON clause - [Issue apache#4097](h2database/h2database#4097): StackOverflowException when using multiple SELECT statements in one query (2.3.230) - [Issue apache#3982](h2database/h2database#3982): Potential issue when using ROUND - [Issue apache#3894](h2database/h2database#3894): Race condition causing stale data in query last result cache - [Issue apache#4075](h2database/h2database#4075): infinite loop in compact - [Issue apache#4091](h2database/h2database#4091): Wrong case with linked table to postgresql - [Issue apache#4088](h2database/h2database#4088): BadGrammarException when the same alias is used within two different CTEs 2. For `postgresql`, there are some issues fixed and improvements in version 42.7.4(full release notes: https://jdbc.postgresql.org/changelogs/2024-08-22-42.7.4-release/): - fix: PgInterval ignores case for represented interval string [PR apache#3344](pgjdbc/pgjdbc#3344) - perf: Avoid extra copies when receiving int4 and int2 in PGStream [PR apache#3295](pgjdbc/pgjdbc#3295) - fix: Add support for Infinity::numeric values in ResultSet.getObject [PR apache#3304](pgjdbc/pgjdbc#3304) - fix: Ensure order of results for getDouble [PR apache#3301](pgjdbc/pgjdbc#3301) - perf: Replace BufferedOutputStream with unsynchronized PgBufferedOutputStream, allow configuring different Java and SO_SNDBUF buffer sizes [PR apache#3248](pgjdbc/pgjdbc#3248) - fix: Fix SSL tests [PR apache#3260](pgjdbc/pgjdbc#3260) - fix: Support bytea in preferQueryMode=simple [PR apache#3243](pgjdbc/pgjdbc#3243) - fix: Fix [Issue apache#3234](pgjdbc/pgjdbc#3234) - Return -1 as update count for stored procedure calls [PR apache#3235](pgjdbc/pgjdbc#3235) - fix: Fix [Issue apache#3224](pgjdbc/pgjdbc#3224) - conversion for TIME ‘24:00’ to LocalTime breaks in binary-mode [PR apache#3225](pgjdbc/pgjdbc#3225) 3. For `mssql`, there are some issues fixed in 12.8.1.jre11(full release notes: https://github.com/microsoft/mssql-jdbc/releases/tag/v12.8.1): - Adjusted DESTINATION_COL_METADATA_LOCK, in SQLServerBulkCopy, so that is properly released in all cases [PR apache#2492](microsoft/mssql-jdbc#2492) - Reverted "Execute Stored Procedures Directly" feature, as well as subsequent changes related to the feature [PR apache#2493](microsoft/mssql-jdbc#2493) - Changed driver behavior to allow prepared statement objects to be reused, preventing a "multiple queries are not allowed" error [PR apache#2494](microsoft/mssql-jdbc#2494) ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass GA. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#47810 from wayneguow/ug_h2. Authored-by: Wei Guo <[email protected]> Signed-off-by: Kent Yao <[email protected]>

Refactors deeply nested FP style code

e833ca4

Avoids converting predicate sets to lists

1bf3258

Improves compareConditions to handle more subtle cases

cd8860b

scwf reviewed Jan 20, 2015
View reviewed changes

asfgit closed this in 8140802 Jan 20, 2015

liancheng deleted the refactor-boolean-simplification branch January 20, 2015 21:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SQL][Minor] Refactors deeply nested FP style code in BooleanSimplification #4091

[SQL][Minor] Refactors deeply nested FP style code in BooleanSimplification #4091

liancheng commented Jan 18, 2015

SparkQA commented Jan 18, 2015

SparkQA commented Jan 18, 2015

AmplabJenkins commented Jan 18, 2015

SparkQA commented Jan 18, 2015

SparkQA commented Jan 18, 2015

SparkQA commented Jan 18, 2015

AmplabJenkins commented Jan 18, 2015

SparkQA commented Jan 18, 2015

AmplabJenkins commented Jan 18, 2015

liancheng commented Jan 20, 2015

scwf Jan 20, 2015

liancheng Jan 20, 2015

liancheng commented Jan 20, 2015

rxin commented Jan 20, 2015

[SQL][Minor] Refactors deeply nested FP style code in BooleanSimplification #4091

[SQL][Minor] Refactors deeply nested FP style code in BooleanSimplification #4091

Conversation

liancheng commented Jan 18, 2015

SparkQA commented Jan 18, 2015

SparkQA commented Jan 18, 2015

AmplabJenkins commented Jan 18, 2015

SparkQA commented Jan 18, 2015

SparkQA commented Jan 18, 2015

SparkQA commented Jan 18, 2015

AmplabJenkins commented Jan 18, 2015

SparkQA commented Jan 18, 2015

AmplabJenkins commented Jan 18, 2015

liancheng commented Jan 20, 2015

scwf Jan 20, 2015

Choose a reason for hiding this comment

liancheng Jan 20, 2015

Choose a reason for hiding this comment

liancheng commented Jan 20, 2015

rxin commented Jan 20, 2015