Skip to content

[SPARK-40540][SQL] Migrate compilation errors onto error classes #37973

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MaxGekk wants to merge 13 commits into from
Closed

[SPARK-40540][SQL] Migrate compilation errors onto error classes #37973

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
4943715
Migrate compilation errors onto error classes
MaxGekk Sep 22, 2022
2f51644
Add error classes 1010-1019
MaxGekk Sep 23, 2022
02e9c62
Convert to error classes: 1020-1029
MaxGekk Sep 23, 2022
5958478
Convert to error classes: 1030-1039
MaxGekk Sep 23, 2022
5e0fbda
Fix tests
MaxGekk Sep 23, 2022
6c79cf2
Add error classes 1040-1049
MaxGekk Sep 23, 2022
91137ab
Merge remote-tracking branch 'origin/master' into legacy-error-temp-c…
MaxGekk Sep 23, 2022
fadd612
Fix tests
MaxGekk Sep 23, 2022
74f7706
Trigger build
MaxGekk Sep 23, 2022
422453b
Add error classes 1060-1069
MaxGekk Sep 24, 2022
00d979f
Add error classes 1070-1079
MaxGekk Sep 24, 2022
84c3805
Add error classes 1079-1099
MaxGekk Sep 24, 2022
52ef0cf
Fix tests
MaxGekk Sep 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
252 changes: 252 additions & 0 deletions core/src/main/resources/error/error-classes.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1163,5 +1163,257 @@
"message" : [
"<msg>."
]
},
"_LEGACY_ERROR_TEMP_1000" : {

@amaliujia amaliujia Sep 23, 2022

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just wondering why we choose use numbers than a descriptive way for the names?

E.g. _LEGACY_ERROR_TEMP_1000 => _LEGACY_ERROR_TEMP_LEGACY_STORE_ASSIGNMENT_POLICY_DISALLOWED?

@amaliujia amaliujia Sep 23, 2022

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it will cause too much work for the migration. If this does not bring much values please feel free to ignore.

@MaxGekk MaxGekk Sep 23, 2022 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just wondering why we choose use numbers ...

@amaliujia Please, see in PR's description:

Why are the changes needed?

The migration on temporary error classes allows to gather statistics about errors and detect most popular error classes.

@MaxGekk MaxGekk Sep 23, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amaliujia Also see #37916 (comment)

@amaliujia amaliujia Sep 23, 2022

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the context! I see it better now.

"message" : [
"LEGACY store assignment policy is disallowed in Spark data source V2. Please set the configuration <configKey> to other values."
]
},
"_LEGACY_ERROR_TEMP_1001" : {
"message" : [
"USING column `<colName>` cannot be resolved on the <side> side of the join. The <side>-side columns: [<plan>]."
]
},
"_LEGACY_ERROR_TEMP_1002" : {
"message" : [
"Unable to generate an encoder for inner class `<className>` without access to the scope that this class was defined in.",
"Try moving this class out of its parent class."
]
},
"_LEGACY_ERROR_TEMP_1003" : {
"message" : [
"Couldn't find the reference column for <after> at <parentName>."
]
},
"_LEGACY_ERROR_TEMP_1004" : {
"message" : [
"Window specification <windowName> is not defined in the WINDOW clause."
]
},
"_LEGACY_ERROR_TEMP_1005" : {
"message" : [
"<expr> doesn't show up in the GROUP BY list <groupByAliases>."
]
},
"_LEGACY_ERROR_TEMP_1006" : {
"message" : [
"Aggregate expression required for pivot, but '<sql>' did not appear in any aggregate function."
]
},
"_LEGACY_ERROR_TEMP_1007" : {
"message" : [
"Cannot write into temp view <quoted> as it's not a data source v2 relation."
]
},
"_LEGACY_ERROR_TEMP_1008" : {
"message" : [
"<quoted> is not a temp view of streaming logical plan, please use batch API such as `DataFrameReader.table` to read it."
]
},
"_LEGACY_ERROR_TEMP_1009" : {
"message" : [
"The depth of view <identifier> exceeds the maximum view resolution depth (<maxNestedViewDepth>). Analysis is aborted to avoid errors. Increase the value of <config> to work around this."
]
},
"_LEGACY_ERROR_TEMP_1010" : {
"message" : [
"Inserting into a view is not allowed. View: <identifier>."
]
},
"_LEGACY_ERROR_TEMP_1011" : {
"message" : [
"Writing into a view is not allowed. View: <identifier>."
]
},
"_LEGACY_ERROR_TEMP_1012" : {
"message" : [
"Cannot write into v1 table: <identifier>."
]
},
"_LEGACY_ERROR_TEMP_1013" : {
"message" : [
"<nameParts> is a <viewStr>. '<cmd>' expects a table.<hintStr>"
]
},
"_LEGACY_ERROR_TEMP_1014" : {
"message" : [
"<nameParts> is a temp view. '<cmd>' expects a permanent view."
]
},
"_LEGACY_ERROR_TEMP_1015" : {
"message" : [
"<identifier> is a table. '<cmd>' expects a view.<hintStr>"
]
},
"_LEGACY_ERROR_TEMP_1016" : {
"message" : [
"<nameParts> is a temp view. '<cmd>' expects a table or permanent view."
]
},
"_LEGACY_ERROR_TEMP_1017" : {
"message" : [
"<name> is a built-in/temporary function. '<cmd>' expects a persistent function.<hintStr>"
]
},
"_LEGACY_ERROR_TEMP_1018" : {
"message" : [
"<quoted> is a permanent view, which is not supported by streaming reading API such as `DataStreamReader.table` yet."
]
},
"_LEGACY_ERROR_TEMP_1019" : {
"message" : [
"Star (*) is not allowed in select list when GROUP BY ordinal position is used"
]
},
"_LEGACY_ERROR_TEMP_1020" : {
"message" : [
"Invalid usage of <elem> in <prettyName>."
]
},
"_LEGACY_ERROR_TEMP_1021" : {
"message" : [
"count(<targetString>.*) is not allowed. Please use count(*) or expand the columns manually, e.g. count(col1, col2)."
]
},
"_LEGACY_ERROR_TEMP_1022" : {
"message" : [
"ORDER BY position <index> is not in select list (valid range is [1, <size>])."
]
},
"_LEGACY_ERROR_TEMP_1023" : {
"message" : [
"Function <prettyName> does not support <syntax>."
]
},
"_LEGACY_ERROR_TEMP_1024" : {
"message" : [
"FILTER expression is non-deterministic, it cannot be used in aggregate functions."
]
},
"_LEGACY_ERROR_TEMP_1025" : {
"message" : [
"FILTER expression is not of type boolean. It cannot be used in an aggregate function."
]
},
"_LEGACY_ERROR_TEMP_1026" : {
"message" : [
"FILTER expression contains aggregate. It cannot be used in an aggregate function."
]
},
"_LEGACY_ERROR_TEMP_1027" : {
"message" : [
"FILTER expression contains window function. It cannot be used in an aggregate function."
]
},
"_LEGACY_ERROR_TEMP_1028" : {
"message" : [
"Number of column aliases does not match number of columns. Number of column aliases: <columnSize>; number of columns: <outputSize>."
]
},
"_LEGACY_ERROR_TEMP_1029" : {
"message" : [
"The number of aliases supplied in the AS clause does not match the number of columns output by the UDTF expected <aliasesSize> aliases but got <aliasesNames>."
]
},
"_LEGACY_ERROR_TEMP_1030" : {
"message" : [
"Window aggregate function with filter predicate is not supported yet."
]
},
"_LEGACY_ERROR_TEMP_1031" : {
"message" : [
"It is not allowed to use a window function inside an aggregate function. Please use the inner window function in a sub-query."
]
},
"_LEGACY_ERROR_TEMP_1032" : {
"message" : [
"<expr> does not have any WindowExpression."
]
},
"_LEGACY_ERROR_TEMP_1033" : {
"message" : [
"<expr> has multiple Window Specifications (<distinctWindowSpec>).",
"Please file a bug report with this error message, stack trace, and the query."
]
},
"_LEGACY_ERROR_TEMP_1034" : {
"message" : [
"It is not allowed to use window functions inside <clauseName> clause."
]
},
"_LEGACY_ERROR_TEMP_1035" : {
"message" : [
"Cannot specify window frame for <prettyName> function."
]
},
"_LEGACY_ERROR_TEMP_1036" : {
"message" : [
"Window Frame <wf> must match the required frame <required>."
]
},
"_LEGACY_ERROR_TEMP_1037" : {
"message" : [
"Window function <wf> requires window to be ordered, please add ORDER BY clause. For example SELECT <wf>(value_expr) OVER (PARTITION BY window_partition ORDER BY window_ordering) from table."
]
},
"_LEGACY_ERROR_TEMP_1038" : {
"message" : [
"Cannot write to table due to mismatched user specified column size(<columnSize>) and data column size(<outputSize>)."
]
},
"_LEGACY_ERROR_TEMP_1039" : {
"message" : [
"Multiple time/session window expressions would result in a cartesian product of rows, therefore they are currently not supported."
]
},
"_LEGACY_ERROR_TEMP_1040" : {
"message" : [
"Gap duration expression used in session window must be CalendarIntervalType, but got <dt>."
]
},
"_LEGACY_ERROR_TEMP_1041" : {
"message" : [
"Undefined function <name>."
]
},
"_LEGACY_ERROR_TEMP_1042" : {
"message" : [
"Invalid number of arguments for function <name>. Expected: <expectedInfo>; Found: <actualNumber>."
]
},
"_LEGACY_ERROR_TEMP_1043" : {
"message" : [
"Invalid arguments for function <name>."
]
},
"_LEGACY_ERROR_TEMP_1044" : {
"message" : [
"Function <name> accepts only one argument."
]
},
"_LEGACY_ERROR_TEMP_1045" : {
"message" : [
"ALTER TABLE SET LOCATION does not support partition for v2 tables."
]
},
"_LEGACY_ERROR_TEMP_1046" : {
"message" : [
"Join strategy hint parameter should be an identifier or string but was <unsupported> (<class>)."
]
},
"_LEGACY_ERROR_TEMP_1047" : {
"message" : [
"<hintName> Hint parameter should include columns, but <invalidParams> found."
]
},
"_LEGACY_ERROR_TEMP_1048" : {
"message" : [
"<hintName> Hint expects a partition number as a parameter."
]
},
"_LEGACY_ERROR_TEMP_1049" : {
"message" : [
"Syntax error in attribute name: <name>."
]
}
}
Loading