Skip to content

[SPARK-41176][SQL] Assign a name to the error class _LEGACY_ERROR_TEMP_1042 #38707

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
panbingkun wants to merge 7 commits into from
Closed

[SPARK-41176][SQL] Assign a name to the error class _LEGACY_ERROR_TEMP_1042 #38707

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3eb543f
[SPARK-41176][SQL] Assign a name to the error class _LEGACY_ERROR_TEM…
panbingkun Nov 18, 2022
247d592
[SPARK-41176][SQL] Assign a name to the error class _LEGACY_ERROR_TEM…
panbingkun Nov 18, 2022
794e99a
[SPARK-41176][SQL] Assign a name to the error class _LEGACY_ERROR_TEM…
panbingkun Nov 19, 2022
c597779
Merge branch 'master' into SPARK-41176
panbingkun Nov 23, 2022
32ef94e
[SPARK-41176][SQL] Assign a name to the error class _LEGACY_ERROR_TEM…
panbingkun Nov 23, 2022
6debe26
[SPARK-41176][SQL] Assign a name to the error class _LEGACY_ERROR_TEM…
panbingkun Nov 23, 2022
5a35eea
[SPARK-41176][SQL] Assign a name to the error class _LEGACY_ERROR_TEM…
panbingkun Nov 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions core/src/main/resources/error/error-classes.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,11 @@
"The operation <statement> is not allowed on the <objectType>: <objectName>"
]
},
"FUNCTION_WRONG_NUM_ARGS" : {
Copy link
Member

@MaxGekk MaxGekk Nov 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cloud-fan @srielau Are you ok with the name or maybe just WRONG_NUM_ARGS? Later we will merge DATATYPE_MISMATCH.WRONG_NUM_ARGS into it.

Copy link
Contributor

@srielau srielau Nov 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WRONG_NUM_ARGs is fine. If we want to keep a longer name I'd prefer ROUTINE over FUNCTIOn for future proofing.

Copy link
Member

@MaxGekk MaxGekk Nov 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@panbingkun Please, rename it to WRONG_NUM_ARGS.

Copy link
Contributor Author

@panbingkun panbingkun Nov 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

"message" : [
"The <functionName> requires <expectedNum> parameters but the actual number is <actualNum>."
]
},
"GRAPHITE_SINK_INVALID_PROTOCOL" : {
"message" : [
"Invalid Graphite protocol: <protocol>"
Expand Down Expand Up @@ -1950,11 +1955,6 @@
"Undefined function <name>."
]
},
"_LEGACY_ERROR_TEMP_1042" : {
"message" : [
"Invalid number of arguments for function <name>. Expected: <expectedInfo>; Found: <actualNumber>."
]
},
"_LEGACY_ERROR_TEMP_1043" : {
"message" : [
"Invalid arguments for function <name>."
Expand Down
13 changes: 6 additions & 7 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -637,13 +637,13 @@ private[sql] object QueryCompilationErrors extends QueryErrorsBase {
}

def invalidFunctionArgumentsError(
name: String, expectedInfo: String, actualNumber: Int): Throwable = {
name: String, expectedNum: String, actualNum: Int): Throwable = {
new AnalysisException(
errorClass = "_LEGACY_ERROR_TEMP_1042",
errorClass = "FUNCTION_WRONG_NUM_ARGS",
messageParameters = Map(
"name" -> name,
"expectedInfo" -> expectedInfo,
"actualNumber" -> actualNumber.toString))
"functionName" -> toSQLId(name),
"expectedNum" -> expectedNum,
"actualNum" -> actualNum.toString))
}

def invalidFunctionArgumentNumberError(
Expand All @@ -656,8 +656,7 @@ private[sql] object QueryCompilationErrors extends QueryErrorsBase {
val expectedNumberOfParameters = if (validParametersCount.length == 1) {
validParametersCount.head.toString
} else {
validParametersCount.init.mkString("one of ", ", ", " and ") +
validParametersCount.last
validParametersCount.mkString("[", ", ", "]")
}
invalidFunctionArgumentsError(name, expectedNumberOfParameters, actualNumber)
}
Expand Down
16 changes: 8 additions & 8 deletions sql/core/src/test/resources/sql-tests/results/ansi/string-functions.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -821,11 +821,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "0",
"expectedInfo" : "2",
"name" : "decode"
"actualNum" : "0",
"expectedNum" : "2",
"functionName" : "`decode`"
},
"queryContext" : [ {
"objectType" : "",
Expand All @@ -844,11 +844,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "1",
"expectedInfo" : "2",
"name" : "decode"
"actualNum" : "1",
"expectedNum" : "2",
"functionName" : "`decode`"
},
"queryContext" : [ {
"objectType" : "",
Expand Down
16 changes: 8 additions & 8 deletions sql/core/src/test/resources/sql-tests/results/ceil-floor-with-scale-param.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,11 +140,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "3",
"expectedInfo" : "2",
"name" : "ceil"
"actualNum" : "3",
"expectedNum" : "2",
"functionName" : "`ceil`"
},
"queryContext" : [ {
"objectType" : "",
Expand Down Expand Up @@ -297,11 +297,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "3",
"expectedInfo" : "2",
"name" : "floor"
"actualNum" : "3",
"expectedNum" : "2",
"functionName" : "`floor`"
},
"queryContext" : [ {
"objectType" : "",
Expand Down
8 changes: 4 additions & 4 deletions sql/core/src/test/resources/sql-tests/results/csv-functions.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,11 +105,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "0",
"expectedInfo" : "one of 2 and 3",
"name" : "from_csv"
"actualNum" : "0",
"expectedNum" : "[2, 3]",
"functionName" : "`from_csv`"
},
"queryContext" : [ {
"objectType" : "",
Expand Down
32 changes: 16 additions & 16 deletions sql/core/src/test/resources/sql-tests/results/json-functions.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,11 +109,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "0",
"expectedInfo" : "one of 1 and 2",
"name" : "to_json"
"actualNum" : "0",
"expectedNum" : "[1, 2]",
"functionName" : "`to_json`"
},
"queryContext" : [ {
"objectType" : "",
Expand Down Expand Up @@ -231,11 +231,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "0",
"expectedInfo" : "one of 2 and 3",
"name" : "from_json"
"actualNum" : "0",
"expectedNum" : "[2, 3]",
"functionName" : "`from_json`"
},
"queryContext" : [ {
"objectType" : "",
Expand Down Expand Up @@ -572,11 +572,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "0",
"expectedInfo" : "1",
"name" : "json_array_length"
"actualNum" : "0",
"expectedNum" : "1",
"functionName" : "`json_array_length`"
},
"queryContext" : [ {
"objectType" : "",
Expand Down Expand Up @@ -659,11 +659,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "0",
"expectedInfo" : "1",
"name" : "json_object_keys"
"actualNum" : "0",
"expectedNum" : "1",
"functionName" : "`json_object_keys`"
},
"queryContext" : [ {
"objectType" : "",
Expand Down
16 changes: 8 additions & 8 deletions sql/core/src/test/resources/sql-tests/results/string-functions.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -753,11 +753,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "0",
"expectedInfo" : "2",
"name" : "decode"
"actualNum" : "0",
"expectedNum" : "2",
"functionName" : "`decode`"
},
"queryContext" : [ {
"objectType" : "",
Expand All @@ -776,11 +776,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "1",
"expectedInfo" : "2",
"name" : "decode"
"actualNum" : "1",
"expectedNum" : "2",
"functionName" : "`decode`"
},
"queryContext" : [ {
"objectType" : "",
Expand Down
2 changes: 1 addition & 1 deletion sql/core/src/test/resources/sql-tests/results/table-valued-functions.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ org.apache.spark.sql.AnalysisException
"errorClass" : "_LEGACY_ERROR_TEMP_1179",
"messageParameters" : {
"arguments" : "integer, integer, integer, integer, integer",
"details" : "Invalid number of arguments for function range. Expected: one of 1, 2, 3 and 4; Found: 5.",
"details" : "[FUNCTION_WRONG_NUM_ARGS] The `range` requires [1, 2, 3, 4] parameters but the actual number is 5.",
"name" : "range",
"usage" : "\n range(start: long, end: long, step: long, numSlices: integer)\n range(start: long, end: long, step: long)\n range(start: long, end: long)\n range(end: long)"
},
Expand Down
8 changes: 4 additions & 4 deletions sql/core/src/test/resources/sql-tests/results/timestamp-ntz.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "7",
"expectedInfo" : "6",
"name" : "make_timestamp_ntz"
"actualNum" : "7",
"expectedNum" : "6",
"functionName" : "`make_timestamp_ntz`"
},
"queryContext" : [ {
"objectType" : "",
Expand Down
8 changes: 4 additions & 4 deletions sql/core/src/test/resources/sql-tests/results/udaf/udaf.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "2",
"expectedInfo" : "1",
"name" : "spark_catalog.default.mydoubleavg"
"actualNum" : "2",
"expectedNum" : "1",
"functionName" : "`spark_catalog.default.mydoubleavg`"
},
"queryContext" : [ {
"objectType" : "",
Expand Down
8 changes: 4 additions & 4 deletions sql/core/src/test/resources/sql-tests/results/udf/udf-udaf.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,11 @@ struct<>
-- !query output
org.apache.spark.sql.AnalysisException
{
"errorClass" : "_LEGACY_ERROR_TEMP_1042",
"errorClass" : "FUNCTION_WRONG_NUM_ARGS",
"messageParameters" : {
"actualNumber" : "2",
"expectedInfo" : "1",
"name" : "spark_catalog.default.mydoubleavg"
"actualNum" : "2",
"expectedNum" : "1",
"functionName" : "`spark_catalog.default.mydoubleavg`"
},
"queryContext" : [ {
"objectType" : "",
Expand Down
37 changes: 29 additions & 8 deletions sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ import org.apache.spark.SparkException
import org.apache.spark.sql.catalyst.InternalRow
import org.apache.spark.sql.catalyst.analysis.{FunctionRegistry, UnresolvedAttribute}
import org.apache.spark.sql.catalyst.expressions.{Alias, ArraysZip, AttributeReference, Expression, NamedExpression, UnaryExpression}
import org.apache.spark.sql.catalyst.expressions.Cast._
import org.apache.spark.sql.catalyst.expressions.codegen.CodegenFallback
import org.apache.spark.sql.catalyst.plans.logical.OneRowRelation
import org.apache.spark.sql.catalyst.util.DateTimeTestUtils.{withDefaultTimeZone, UTC}
Expand Down Expand Up @@ -4847,14 +4848,34 @@ class DataFrameFunctionsSuite extends QueryTest with SharedSparkSession {
(Seq.empty, Seq("x", "z"), 3),
(null, Seq("x", "z"), 4)
).toDF("a1", "a2", "i")
val ex1 = intercept[AnalysisException] {
df.selectExpr("zip_with(a1, a2, x -> x)")
}
assert(ex1.getMessage.contains("The number of lambda function arguments '1' does not match"))
val ex2 = intercept[AnalysisException] {
df.selectExpr("zip_with(a1, a2, (acc, x) -> x, (acc, x) -> x)")
}
assert(ex2.getMessage.contains("Invalid number of arguments for function zip_with"))
checkError(
exception = intercept[AnalysisException] {
df.selectExpr("zip_with(a1, a2, x -> x)")
},
errorClass = "_LEGACY_ERROR_TEMP_2300",
parameters = Map(
"namesSize" -> "1",
"argInfoSize" -> "2"),
context = ExpectedContext(
fragment = "x -> x",
start = 17,
stop = 22)
)

checkError(
exception = intercept[AnalysisException] {
df.selectExpr("zip_with(a1, a2, (acc, x) -> x, (acc, x) -> x)")
},
errorClass = "FUNCTION_WRONG_NUM_ARGS",
parameters = Map(
"functionName" -> toSQLId("zip_with"),
"expectedNum" -> "3",
"actualNum" -> "4"),
context = ExpectedContext(
fragment = "zip_with(a1, a2, (acc, x) -> x, (acc, x) -> x)",
start = 0,
stop = 45)
)

checkError(
exception = intercept[AnalysisException] {
Expand Down
18 changes: 16 additions & 2 deletions sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2630,8 +2630,22 @@ class SQLQuerySuite extends QueryTest with SharedSparkSession with AdaptiveSpark
}

test("RuntimeReplaceable functions should not take extra parameters") {
val e = intercept[AnalysisException](sql("SELECT nvl(1, 2, 3)"))
assert(e.message.contains("Invalid number of arguments"))
checkError(
exception = intercept[AnalysisException] {
sql("SELECT nvl(1, 2, 3)")
},
errorClass = "FUNCTION_WRONG_NUM_ARGS",
parameters = Map(
"functionName" -> toSQLId("nvl"),
"expectedNum" -> "2",
"actualNum" -> "3"
),
context = ExpectedContext(
start = 7,
stop = 18,
fragment = "nvl(1, 2, 3)"
)
)
}

test("SPARK-21228: InSet incorrect handling of structs") {
Expand Down
Loading