Skip to content

[SPARK-41062][SQL] Rename UNSUPPORTED_CORRELATED_REFERENCE to CORRELATED_REFERENCE #38576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
itholic wants to merge 14 commits into from
Closed

[SPARK-41062][SQL] Rename UNSUPPORTED_CORRELATED_REFERENCE to CORRELATED_REFERENCE #38576

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
9e05a85
[SPARK-41062][SQL] Rename UNSUPPORTED_CORRELATED_REFERENCE to CORRELA…
itholic Nov 9, 2022
68c32b8
Use sqlExpr rather than treeNode
itholic Nov 16, 2022
0ba23c5
fix test
itholic Nov 16, 2022
be182b4
fix test
itholic Nov 16, 2022
a89cefe
Merge branch 'master' of https://github.com/apache/spark into SPARK-4…
itholic Nov 17, 2022
e4869a4
check containsOuter
itholic Nov 17, 2022
d5be2e5
resolved the comments
itholic Nov 22, 2022
1106e0b
fix test
itholic Nov 22, 2022
e2e5cd8
Merge branch 'master' of https://github.com/apache/spark into SPARK-4…
itholic Nov 23, 2022
be2dbe9
fix test
itholic Nov 23, 2022
eaa1b37
Use matchPvals for test
itholic Nov 23, 2022
db6bbef
resolved the conflicts
itholic Dec 1, 2022
89c2a80
Merge branch 'master' of https://github.com/apache/spark into SPARK-4…
itholic Dec 1, 2022
6fcb78f
Update core/src/main/resources/error/error-classes.json
itholic Dec 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions core/src/main/resources/error/error-classes.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1277,6 +1277,11 @@
"A correlated outer name reference within a subquery expression body was not found in the enclosing query: <value>"
]
},
"CORRELATED_REFERENCE" : {
"message" : [
"Expressions referencing the outer query are not supported outside of WHERE/HAVING clauses<treeNode>"
Copy link
Member

@MaxGekk MaxGekk Nov 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's don't confuse users by treeNode, and show something more useful like problematic SQL expr.

Copy link
Contributor

@srielau srielau Nov 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, let's show the expression. (having the expression in the context would also be acceptable). No treenode

Copy link
Contributor Author

@itholic itholic Nov 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaxGekk May I ask if there is any way to extract the SQL expr from LogicalPlan ??

I'm trying to switch planToString(p) to SQL expression but still not sure how.

Copy link
Member

@MaxGekk MaxGekk Nov 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expressions referencing the outer query ...

Can you show the expression?

Copy link
Contributor Author

@itholic itholic Nov 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, just fixed it to show SQL expression!

]
},
"LATERAL_JOIN_CONDITION_NON_DETERMINISTIC" : {
"message" : [
"Lateral join condition cannot be non-deterministic: <condition>"
Expand All @@ -1302,11 +1307,6 @@
"Non-deterministic lateral subqueries are not supported when joining with outer relations that produce more than one row<treeNode>"
]
},
"UNSUPPORTED_CORRELATED_REFERENCE" : {
"message" : [
"Expressions referencing the outer query are not supported outside of WHERE/HAVING clauses<treeNode>"
]
},
"UNSUPPORTED_CORRELATED_REFERENCE_DATA_TYPE" : {
"message" : [
"Correlated column reference '<expr>' cannot be <dataType> type"
Expand Down
2 changes: 1 addition & 1 deletion sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1059,7 +1059,7 @@ trait CheckAnalysis extends PredicateHelper with LookupCatalog with QueryErrorsB
if (!canHostOuter(p) && p.expressions.exists(containsOuter)) {
p.failAnalysis(
errorClass =
"UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.UNSUPPORTED_CORRELATED_REFERENCE",
"UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.CORRELATED_REFERENCE",
messageParameters = Map("treeNode" -> planToString(p)))
}
}
Expand Down
2 changes: 1 addition & 1 deletion .../src/test/resources/sql-tests/results/subquery/negative-cases/invalid-correlation.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.UNSUPPORTED_CORRELATED_REFERENCE",
"errorClass" : "UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.CORRELATED_REFERENCE",
"messageParameters" : {
"treeNode" : "Aggregate [min(outer(t2a#x)) AS min(outer(t2.t2a))#x]\n+- SubqueryAlias t3\n +- View (`t3`, [t3a#x,t3b#x,t3c#x])\n +- Project [cast(t3a#x as int) AS t3a#x, cast(t3b#x as int) AS t3b#x, cast(t3c#x as int) AS t3c#x]\n +- Project [t3a#x, t3b#x, t3c#x]\n +- SubqueryAlias t3\n +- LocalRelation [t3a#x, t3b#x, t3c#x]\n"
},
Expand Down
2 changes: 1 addition & 1 deletion sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -966,7 +966,7 @@ class SubquerySuite extends QueryTest
}
checkErrorMatchPVals(
exception1,
errorClass = "UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.UNSUPPORTED_CORRELATED_REFERENCE",
errorClass = "UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.CORRELATED_REFERENCE",
parameters = Map("treeNode" -> "(?s).*"),
sqlState = None,
context = ExpectedContext(
Expand Down