Skip to content

[SPARK-34718][SQL] Assign pretty names to YearMonthIntervalType and DayTimeIntervalType #31810

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MaxGekk wants to merge 2 commits into from
Closed

[SPARK-34718][SQL] Assign pretty names to YearMonthIntervalType and DayTimeIntervalType #31810

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8fc9896
Override typeName
MaxGekk Mar 11, 2021
9755022
Fix expected error messages
MaxGekk Mar 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/types/DayTimeIntervalType.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ class DayTimeIntervalType private() extends AtomicType {
override def defaultSize: Int = 8

private[spark] override def asNullable: DayTimeIntervalType = this

override def typeName: String = "day-time interval"
}

/**
Expand Down
2 changes: 2 additions & 0 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/types/YearMonthIntervalType.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ class YearMonthIntervalType private() extends AtomicType {
override def defaultSize: Int = 4

private[spark] override def asNullable: YearMonthIntervalType = this

override def typeName: String = "year-month interval"
Copy link
Member

@yaooqinn yaooqinn Mar 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

according to the standard，year-month and day-time is italic，so I guess interval here is unnecessary.

Copy link
Member Author

@MaxGekk MaxGekk Mar 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I haven't found any places in the SQL standard where year-month and day-time are used without the interval(s) word(s).
  2. Actual type name is INTERVAL, year-month and day-time just define (sub-)classes of the type.
  3. Since the typeName() method is used in error messages, I do believe we should leave interval in type names for readability.

}

/**
Expand Down
4 changes: 2 additions & 2 deletions ...t/src/test/scala/org/apache/spark/sql/catalyst/analysis/ExpressionTypeCheckingSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,9 @@ class ExpressionTypeCheckingSuite extends SparkFunSuite {
assertErrorForDifferingTypes(BitwiseXor(Symbol("intField"), Symbol("booleanField")))

assertError(Add(Symbol("booleanField"), Symbol("booleanField")),
"requires (numeric or interval or daytimeinterval or yearmonthinterval) type")
"requires (numeric or interval or day-time interval or year-month interval) type")
assertError(Subtract(Symbol("booleanField"), Symbol("booleanField")),
"requires (numeric or interval or daytimeinterval or yearmonthinterval) type")
"requires (numeric or interval or day-time interval or year-month interval) type")
assertError(Multiply(Symbol("booleanField"), Symbol("booleanField")), "requires numeric type")
assertError(Divide(Symbol("booleanField"), Symbol("booleanField")),
"requires (double or decimal) type")
Expand Down
18 changes: 9 additions & 9 deletions sql/core/src/test/resources/sql-tests/results/ansi/literals.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -436,7 +436,7 @@ select +date '1999-01-01'
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(+ DATE '1999-01-01')' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'DATE '1999-01-01'' is of date type.; line 1 pos 7
cannot resolve '(+ DATE '1999-01-01')' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'DATE '1999-01-01'' is of date type.; line 1 pos 7


-- !query
Expand All @@ -445,7 +445,7 @@ select +timestamp '1999-01-01'
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(+ TIMESTAMP '1999-01-01 00:00:00')' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'TIMESTAMP '1999-01-01 00:00:00'' is of timestamp type.; line 1 pos 7
cannot resolve '(+ TIMESTAMP '1999-01-01 00:00:00')' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'TIMESTAMP '1999-01-01 00:00:00'' is of timestamp type.; line 1 pos 7


-- !query
Expand All @@ -462,7 +462,7 @@ select +map(1, 2)
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(+ map(1, 2))' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'map(1, 2)' is of map<int,int> type.; line 1 pos 7
cannot resolve '(+ map(1, 2))' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'map(1, 2)' is of map<int,int> type.; line 1 pos 7


-- !query
Expand All @@ -471,7 +471,7 @@ select +array(1,2)
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(+ array(1, 2))' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'array(1, 2)' is of array<int> type.; line 1 pos 7
cannot resolve '(+ array(1, 2))' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'array(1, 2)' is of array<int> type.; line 1 pos 7


-- !query
Expand All @@ -480,7 +480,7 @@ select +named_struct('a', 1, 'b', 'spark')
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(+ named_struct('a', 1, 'b', 'spark'))' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'named_struct('a', 1, 'b', 'spark')' is of struct<a:int,b:string> type.; line 1 pos 7
cannot resolve '(+ named_struct('a', 1, 'b', 'spark'))' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'named_struct('a', 1, 'b', 'spark')' is of struct<a:int,b:string> type.; line 1 pos 7


-- !query
Expand All @@ -489,7 +489,7 @@ select +X'1'
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(+ X'01')' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'X'01'' is of binary type.; line 1 pos 7
cannot resolve '(+ X'01')' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'X'01'' is of binary type.; line 1 pos 7


-- !query
Expand All @@ -498,7 +498,7 @@ select -date '1999-01-01'
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(- DATE '1999-01-01')' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'DATE '1999-01-01'' is of date type.; line 1 pos 7
cannot resolve '(- DATE '1999-01-01')' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'DATE '1999-01-01'' is of date type.; line 1 pos 7


-- !query
Expand All @@ -507,7 +507,7 @@ select -timestamp '1999-01-01'
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(- TIMESTAMP '1999-01-01 00:00:00')' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'TIMESTAMP '1999-01-01 00:00:00'' is of timestamp type.; line 1 pos 7
cannot resolve '(- TIMESTAMP '1999-01-01 00:00:00')' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'TIMESTAMP '1999-01-01 00:00:00'' is of timestamp type.; line 1 pos 7


-- !query
Expand All @@ -516,4 +516,4 @@ select -x'2379ACFe'
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(- X'2379ACFE')' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'X'2379ACFE'' is of binary type.; line 1 pos 7
cannot resolve '(- X'2379ACFE')' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'X'2379ACFE'' is of binary type.; line 1 pos 7
Expand Down
18 changes: 9 additions & 9 deletions sql/core/src/test/resources/sql-tests/results/literals.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -436,7 +436,7 @@ select +date '1999-01-01'
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(+ DATE '1999-01-01')' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'DATE '1999-01-01'' is of date type.; line 1 pos 7
cannot resolve '(+ DATE '1999-01-01')' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'DATE '1999-01-01'' is of date type.; line 1 pos 7


-- !query
Expand All @@ -445,7 +445,7 @@ select +timestamp '1999-01-01'
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(+ TIMESTAMP '1999-01-01 00:00:00')' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'TIMESTAMP '1999-01-01 00:00:00'' is of timestamp type.; line 1 pos 7
cannot resolve '(+ TIMESTAMP '1999-01-01 00:00:00')' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'TIMESTAMP '1999-01-01 00:00:00'' is of timestamp type.; line 1 pos 7


-- !query
Expand All @@ -462,7 +462,7 @@ select +map(1, 2)
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(+ map(1, 2))' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'map(1, 2)' is of map<int,int> type.; line 1 pos 7
cannot resolve '(+ map(1, 2))' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'map(1, 2)' is of map<int,int> type.; line 1 pos 7


-- !query
Expand All @@ -471,7 +471,7 @@ select +array(1,2)
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(+ array(1, 2))' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'array(1, 2)' is of array<int> type.; line 1 pos 7
cannot resolve '(+ array(1, 2))' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'array(1, 2)' is of array<int> type.; line 1 pos 7


-- !query
Expand All @@ -480,7 +480,7 @@ select +named_struct('a', 1, 'b', 'spark')
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(+ named_struct('a', 1, 'b', 'spark'))' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'named_struct('a', 1, 'b', 'spark')' is of struct<a:int,b:string> type.; line 1 pos 7
cannot resolve '(+ named_struct('a', 1, 'b', 'spark'))' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'named_struct('a', 1, 'b', 'spark')' is of struct<a:int,b:string> type.; line 1 pos 7


-- !query
Expand All @@ -489,7 +489,7 @@ select +X'1'
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(+ X'01')' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'X'01'' is of binary type.; line 1 pos 7
cannot resolve '(+ X'01')' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'X'01'' is of binary type.; line 1 pos 7


-- !query
Expand All @@ -498,7 +498,7 @@ select -date '1999-01-01'
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(- DATE '1999-01-01')' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'DATE '1999-01-01'' is of date type.; line 1 pos 7
cannot resolve '(- DATE '1999-01-01')' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'DATE '1999-01-01'' is of date type.; line 1 pos 7


-- !query
Expand All @@ -507,7 +507,7 @@ select -timestamp '1999-01-01'
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(- TIMESTAMP '1999-01-01 00:00:00')' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'TIMESTAMP '1999-01-01 00:00:00'' is of timestamp type.; line 1 pos 7
cannot resolve '(- TIMESTAMP '1999-01-01 00:00:00')' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'TIMESTAMP '1999-01-01 00:00:00'' is of timestamp type.; line 1 pos 7


-- !query
Expand All @@ -516,4 +516,4 @@ select -x'2379ACFe'
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve '(- X'2379ACFE')' due to data type mismatch: argument 1 requires (numeric or interval or daytimeinterval or yearmonthinterval) type, however, 'X'2379ACFE'' is of binary type.; line 1 pos 7
cannot resolve '(- X'2379ACFE')' due to data type mismatch: argument 1 requires (numeric or interval or day-time interval or year-month interval) type, however, 'X'2379ACFE'' is of binary type.; line 1 pos 7
Expand Down
6 changes: 3 additions & 3 deletions ...core/src/test/resources/sql-tests/results/typeCoercion/native/windowFrameCoercion.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ SELECT COUNT(*) OVER (PARTITION BY 1 ORDER BY cast(1 as string) DESC RANGE BETWE
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve 'RANGE BETWEEN CURRENT ROW AND CAST(1 AS STRING) FOLLOWING' due to data type mismatch: The data type of the upper bound 'string' does not match the expected data type '(numeric or interval or daytimeinterval or yearmonthinterval)'.; line 1 pos 21
cannot resolve 'RANGE BETWEEN CURRENT ROW AND CAST(1 AS STRING) FOLLOWING' due to data type mismatch: The data type of the upper bound 'string' does not match the expected data type '(numeric or interval or day-time interval or year-month interval)'.; line 1 pos 21


-- !query
Expand All @@ -177,7 +177,7 @@ SELECT COUNT(*) OVER (PARTITION BY 1 ORDER BY cast('1' as binary) DESC RANGE BET
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve 'RANGE BETWEEN CURRENT ROW AND CAST(1 AS BINARY) FOLLOWING' due to data type mismatch: The data type of the upper bound 'binary' does not match the expected data type '(numeric or interval or daytimeinterval or yearmonthinterval)'.; line 1 pos 21
cannot resolve 'RANGE BETWEEN CURRENT ROW AND CAST(1 AS BINARY) FOLLOWING' due to data type mismatch: The data type of the upper bound 'binary' does not match the expected data type '(numeric or interval or day-time interval or year-month interval)'.; line 1 pos 21


-- !query
Expand All @@ -186,7 +186,7 @@ SELECT COUNT(*) OVER (PARTITION BY 1 ORDER BY cast(1 as boolean) DESC RANGE BETW
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
cannot resolve 'RANGE BETWEEN CURRENT ROW AND CAST(1 AS BOOLEAN) FOLLOWING' due to data type mismatch: The data type of the upper bound 'boolean' does not match the expected data type '(numeric or interval or daytimeinterval or yearmonthinterval)'.; line 1 pos 21
cannot resolve 'RANGE BETWEEN CURRENT ROW AND CAST(1 AS BOOLEAN) FOLLOWING' due to data type mismatch: The data type of the upper bound 'boolean' does not match the expected data type '(numeric or interval or day-time interval or year-month interval)'.; line 1 pos 21


-- !query
Expand Down