Skip to content

Commit 58cb2ba

Browse files
belieferHyukjinKwon
beliefer
authored and
HyukjinKwon
committed
[SPARK-33752][SQL] Avoid the getSimpleMessage of AnalysisException adds semicolon repeatedly
### What changes were proposed in this pull request? The current `getSimpleMessage` of `AnalysisException` may adds semicolon repeatedly. There show an example below: `select decode()` The output will be: ``` org.apache.spark.sql.AnalysisException Invalid number of arguments for function decode. Expected: 2; Found: 0;; line 1 pos 7 ``` ### Why are the changes needed? Fix a bug, because it adds semicolon repeatedly. ### Does this PR introduce _any_ user-facing change? Yes. the message of AnalysisException will be correct. ### How was this patch tested? Jenkins test. Closes #30724 from beliefer/SPARK-33752. Lead-authored-by: gengjiaan <[email protected]> Co-authored-by: beliefer <[email protected]> Signed-off-by: HyukjinKwon <[email protected]>
1 parent 20f6d63 commit 58cb2ba

File tree

60 files changed

+264
-265
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+264
-265
lines changed

‎sql/catalyst/src/main/scala/org/apache/spark/sql/AnalysisException.scala‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,9 +48,11 @@ class AnalysisException protected[sql] (
4848

4949
// Outputs an exception without the logical plan.
5050
// For testing only
51-
def getSimpleMessage: String = {
51+
def getSimpleMessage: String = if (line.isDefined || startPosition.isDefined) {
5252
val lineAnnotation = line.map(l => s" line $l").getOrElse("")
5353
val positionAnnotation = startPosition.map(p => s" pos $p").getOrElse("")
5454
s"$message;$lineAnnotation$positionAnnotation"
55+
} else {
56+
message
5557
}
5658
}

‎sql/core/src/test/resources/sql-tests/results/ansi/datetime.sql.out‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -453,7 +453,7 @@ select date_add('2011-11-11', '1.2')
453453
struct<>
454454
-- !query output
455455
org.apache.spark.sql.AnalysisException
456-
The second argument of 'date_add' function needs to be an integer.;
456+
The second argument of 'date_add' function needs to be an integer.
457457

458458

459459
-- !query
@@ -494,7 +494,7 @@ select date_sub(date'2011-11-11', '1.2')
494494
struct<>
495495
-- !query output
496496
org.apache.spark.sql.AnalysisException
497-
The second argument of 'date_sub' function needs to be an integer.;
497+
The second argument of 'date_sub' function needs to be an integer.
498498

499499

500500
-- !query

‎sql/core/src/test/resources/sql-tests/results/ansi/parse-schema-string.sql.out‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ no viable alternative at input 'create'(line 1, pos 0)
2121
== SQL ==
2222
create INT
2323
^^^
24-
;; line 1 pos 7
24+
; line 1 pos 7
2525

2626

2727
-- !query
@@ -51,7 +51,7 @@ no viable alternative at input 'create'(line 1, pos 0)
5151
== SQL ==
5252
create INT
5353
^^^
54-
;; line 1 pos 7
54+
; line 1 pos 7
5555

5656

5757
-- !query

‎sql/core/src/test/resources/sql-tests/results/ansi/string-functions.sql.out‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -302,7 +302,7 @@ select decode()
302302
struct<>
303303
-- !query output
304304
org.apache.spark.sql.AnalysisException
305-
Invalid number of arguments for function decode. Expected: 2; Found: 0;; line 1 pos 7
305+
Invalid number of arguments for function decode. Expected: 2; Found: 0; line 1 pos 7
306306

307307

308308
-- !query
@@ -311,7 +311,7 @@ select decode(encode('abc', 'utf-8'))
311311
struct<>
312312
-- !query output
313313
org.apache.spark.sql.AnalysisException
314-
Invalid number of arguments for function decode. Expected: 2; Found: 1;; line 1 pos 7
314+
Invalid number of arguments for function decode. Expected: 2; Found: 1; line 1 pos 7
315315

316316

317317
-- !query
@@ -359,4 +359,4 @@ select decode(6, 1, 'Southlake', 2, 'San Francisco', 3, 'New Jersey', 4, 'Seattl
359359
-- !query schema
360360
struct<decode(6, 1, Southlake, 2, San Francisco, 3, New Jersey, 4, Seattle):string>
361361
-- !query output
362-
NULL
362+
NULL

‎sql/core/src/test/resources/sql-tests/results/change-column.sql.out‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ ALTER TABLE test_change RENAME COLUMN a TO a1
5050
struct<>
5151
-- !query output
5252
org.apache.spark.sql.AnalysisException
53-
RENAME COLUMN is only supported with v2 tables.;
53+
RENAME COLUMN is only supported with v2 tables.
5454

5555

5656
-- !query
@@ -69,7 +69,7 @@ ALTER TABLE test_change CHANGE a TYPE STRING
6969
struct<>
7070
-- !query output
7171
org.apache.spark.sql.AnalysisException
72-
ALTER TABLE CHANGE COLUMN is not supported for changing column 'a' with type 'IntegerType' to 'a' with type 'StringType';
72+
ALTER TABLE CHANGE COLUMN is not supported for changing column 'a' with type 'IntegerType' to 'a' with type 'StringType'
7373

7474

7575
-- !query
@@ -88,7 +88,7 @@ ALTER TABLE test_change CHANGE a AFTER b
8888
struct<>
8989
-- !query output
9090
org.apache.spark.sql.AnalysisException
91-
ALTER COLUMN ... FIRST | ALTER is only supported with v2 tables.;
91+
ALTER COLUMN ... FIRST | ALTER is only supported with v2 tables.
9292

9393

9494
-- !query
@@ -97,7 +97,7 @@ ALTER TABLE test_change CHANGE b FIRST
9797
struct<>
9898
-- !query output
9999
org.apache.spark.sql.AnalysisException
100-
ALTER COLUMN ... FIRST | ALTER is only supported with v2 tables.;
100+
ALTER COLUMN ... FIRST | ALTER is only supported with v2 tables.
101101

102102

103103
-- !query
@@ -176,7 +176,7 @@ ALTER TABLE test_change CHANGE invalid_col TYPE INT
176176
struct<>
177177
-- !query output
178178
org.apache.spark.sql.AnalysisException
179-
Can't find column `invalid_col` given table data columns [`a`, `b`, `c`];
179+
Can't find column `invalid_col` given table data columns [`a`, `b`, `c`]
180180

181181

182182
-- !query

‎sql/core/src/test/resources/sql-tests/results/columnresolution-negative.sql.out‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -195,7 +195,7 @@ SELECT t1.x.y.* FROM t1
195195
struct<>
196196
-- !query output
197197
org.apache.spark.sql.AnalysisException
198-
cannot resolve 't1.x.y.*' given input columns 'i1';
198+
cannot resolve 't1.x.y.*' given input columns 'i1'
199199

200200

201201
-- !query

‎sql/core/src/test/resources/sql-tests/results/count.sql.out‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -125,4 +125,4 @@ SELECT count() FROM testData
125125
struct<>
126126
-- !query output
127127
org.apache.spark.sql.AnalysisException
128-
cannot resolve 'count()' due to data type mismatch: count requires at least one argument.; line 1 pos 7
128+
cannot resolve 'count()' due to data type mismatch: count requires at least one argument.; line 1 pos 7

‎sql/core/src/test/resources/sql-tests/results/csv-functions.sql.out‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ select from_csv('1', 1)
2424
struct<>
2525
-- !query output
2626
org.apache.spark.sql.AnalysisException
27-
The expression '1' is not a valid schema string.;; line 1 pos 7
27+
The expression '1' is not a valid schema string.; line 1 pos 7
2828

2929

3030
-- !query
@@ -46,7 +46,7 @@ DataType invalidtype is not supported.(line 1, pos 2)
4646
== SQL ==
4747
a InvalidType
4848
--^^^
49-
;; line 1 pos 7
49+
; line 1 pos 7
5050

5151

5252
-- !query
@@ -55,7 +55,7 @@ select from_csv('1', 'a INT', named_struct('mode', 'PERMISSIVE'))
5555
struct<>
5656
-- !query output
5757
org.apache.spark.sql.AnalysisException
58-
Must use a map() function for options;; line 1 pos 7
58+
Must use a map() function for options; line 1 pos 7
5959

6060

6161
-- !query
@@ -64,7 +64,7 @@ select from_csv('1', 'a INT', map('mode', 1))
6464
struct<>
6565
-- !query output
6666
org.apache.spark.sql.AnalysisException
67-
A type of keys and values in map() must be string, but got map<string,int>;; line 1 pos 7
67+
A type of keys and values in map() must be string, but got map<string,int>; line 1 pos 7
6868

6969

7070
-- !query
@@ -148,7 +148,7 @@ select to_csv(named_struct('a', 1, 'b', 2), named_struct('mode', 'PERMISSIVE'))
148148
struct<>
149149
-- !query output
150150
org.apache.spark.sql.AnalysisException
151-
Must use a map() function for options;; line 1 pos 7
151+
Must use a map() function for options; line 1 pos 7
152152

153153

154154
-- !query
@@ -157,4 +157,4 @@ select to_csv(named_struct('a', 1, 'b', 2), map('mode', 1))
157157
struct<>
158158
-- !query output
159159
org.apache.spark.sql.AnalysisException
160-
A type of keys and values in map() must be string, but got map<string,int>;; line 1 pos 7
160+
A type of keys and values in map() must be string, but got map<string,int>; line 1 pos 7

‎sql/core/src/test/resources/sql-tests/results/cte-nested.sql.out‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ SELECT * FROM t2
4848
struct<>
4949
-- !query output
5050
org.apache.spark.sql.AnalysisException
51-
Name t is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.;
51+
Name t is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.
5252

5353

5454
-- !query
@@ -85,7 +85,7 @@ SELECT * FROM t2
8585
struct<>
8686
-- !query output
8787
org.apache.spark.sql.AnalysisException
88-
Name t is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.;
88+
Name t is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.
8989

9090

9191
-- !query
@@ -139,7 +139,7 @@ SELECT (
139139
struct<>
140140
-- !query output
141141
org.apache.spark.sql.AnalysisException
142-
Name t is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.;
142+
Name t is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.
143143

144144

145145
-- !query
@@ -154,7 +154,7 @@ SELECT (
154154
struct<>
155155
-- !query output
156156
org.apache.spark.sql.AnalysisException
157-
Name t is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.;
157+
Name t is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.
158158

159159

160160
-- !query
@@ -170,7 +170,7 @@ SELECT (
170170
struct<>
171171
-- !query output
172172
org.apache.spark.sql.AnalysisException
173-
Name t is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.;
173+
Name t is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.
174174

175175

176176
-- !query
@@ -184,7 +184,7 @@ WHERE c IN (
184184
struct<>
185185
-- !query output
186186
org.apache.spark.sql.AnalysisException
187-
Name t is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.;
187+
Name t is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.
188188

189189

190190
-- !query
@@ -213,7 +213,7 @@ SELECT * FROM t
213213
struct<>
214214
-- !query output
215215
org.apache.spark.sql.AnalysisException
216-
Name aBc is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.;
216+
Name aBc is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.
217217

218218

219219
-- !query
@@ -226,4 +226,4 @@ SELECT (
226226
struct<>
227227
-- !query output
228228
org.apache.spark.sql.AnalysisException
229-
Name aBc is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.;
229+
Name aBc is ambiguous in nested CTE. Please set spark.sql.legacy.ctePrecedencePolicy to CORRECTED so that name defined in inner CTE takes precedence. If set it to LEGACY, outer CTE definitions will take precedence. See more details in SPARK-28228.

‎sql/core/src/test/resources/sql-tests/results/datetime-legacy.sql.out‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -430,7 +430,7 @@ select date_add('2011-11-11', '1.2')
430430
struct<>
431431
-- !query output
432432
org.apache.spark.sql.AnalysisException
433-
The second argument of 'date_add' function needs to be an integer.;
433+
The second argument of 'date_add' function needs to be an integer.
434434

435435

436436
-- !query
@@ -471,7 +471,7 @@ select date_sub(date'2011-11-11', '1.2')
471471
struct<>
472472
-- !query output
473473
org.apache.spark.sql.AnalysisException
474-
The second argument of 'date_sub' function needs to be an integer.;
474+
The second argument of 'date_sub' function needs to be an integer.
475475

476476

477477
-- !query

0 commit comments

Comments
 (0)