Skip to content

[SPARK-46519][SQL] Clear unused error classes from error-classes.json file #44503

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
panbingkun wants to merge 5 commits into from
Closed

[SPARK-46519][SQL] Clear unused error classes from error-classes.json file #44503

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
286dfdc
[SPARK-46519][SQL] Clear unused error classes from error-classes.json…
panbingkun Dec 27, 2023
9f5bab1
[SPARK-46519][SQL] Clear unused error classes from error-classes.json…
panbingkun Dec 27, 2023
557713d
[SPARK-46519][SQL] Clear unused error classes from error-classes.json…
panbingkun Dec 27, 2023
72e223f
[SPARK-46519][SQL] Clear unused error classes from error-classes.json…
panbingkun Dec 28, 2023
3578d3d
Merge branch 'master' into SPARK-46519
panbingkun Dec 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 0 additions & 32 deletions common/utils/src/main/resources/error/error-classes.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -875,12 +875,6 @@
],
"sqlState" : "42K01"
},
"DATA_SOURCE_ALREADY_EXISTS" : {
"message" : [
"Data source '<provider>' already exists in the registry. Please use a different name for the new data source."
],
"sqlState" : "42710"
},
"DATA_SOURCE_NOT_EXIST" : {
"message" : [
"Data source '<provider>' not found. Please make sure the data source is registered."
Expand Down Expand Up @@ -1480,12 +1474,6 @@
},
"sqlState" : "42K0B"
},
"INCORRECT_END_OFFSET" : {
Copy link
Member

@MaxGekk MaxGekk Dec 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reasons there is still a reference to the error class:

$ find . -type f  -print0|xargs -0 grep INCORRECT_END_OFFSET
./docs/sql-error-conditions-sqlstates.md:  <td><a href="arithmetic-overflow-error-class.md">ARITHMETIC_OVERFLOW</a>, <a href="sql-error-conditions.html#cast_overflow">CAST_OVERFLOW</a>, <a href="sql-error-conditions.html#cast_overflow_in_table_insert">CAST_OVERFLOW_IN_TABLE_INSERT</a>, <a href="sql-error-conditions.html#decimal_precision_exceeds_max_precision">DECIMAL_PRECISION_EXCEEDS_MAX_PRECISION</a>, <a href="sql-error-conditions.html#invalid_index_of_zero">INVALID_INDEX_OF_ZERO</a>, <a href="sql-error-conditions.html#incorrect_end_offset">INCORRECT_END_OFFSET</a>, <a href="sql-error-conditions.html#incorrect_ramp_up_rate">INCORRECT_RAMP_UP_RATE</a>, <a href="invalid-array-index-error-class.md">INVALID_ARRAY_INDEX</a>, <a href="invalid-array-index-in-element-at-error-class.md">INVALID_ARRAY_INDEX_IN_ELEMENT_AT</a>, <a href="sql-error-conditions.html#numeric_out_of_supported_range">NUMERIC_OUT_OF_SUPPORTED_RANGE</a>, <a href="sql-error-conditions.html#numeric_value_out_of_range">NUMERIC_VALUE_OUT_OF_RANGE</a>

Copy link
Member

@MaxGekk MaxGekk Dec 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, remove the reference to the deleted error class.

Copy link
Contributor Author

@panbingkun panbingkun Dec 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, let me delete it first, and then I'll try to investigate the root cause again to see if we can automate its discovery.

Copy link
Contributor Author

@panbingkun panbingkun Dec 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

"message" : [
"Max offset with <rowsPerSecond> rowsPerSecond is <maxSeconds>, but it's <endSeconds> now."
],
"sqlState" : "22003"
},
"INCORRECT_RAMP_UP_RATE" : {
"message" : [
"Max offset with <rowsPerSecond> rowsPerSecond is <maxSeconds>, but 'rampUpTimeSeconds' is <rampUpTimeSeconds>."
Expand Down Expand Up @@ -1906,11 +1894,6 @@
"Operation not found."
]
},
"SESSION_ALREADY_EXISTS" : {
"message" : [
"Session already exists."
]
},
"SESSION_CLOSED" : {
"message" : [
"Session was closed."
Expand Down Expand Up @@ -6065,11 +6048,6 @@
"<walkedTypePath>."
]
},
"_LEGACY_ERROR_TEMP_2142" : {
"message" : [
"Attributes for type <schema> is not supported."
]
},
"_LEGACY_ERROR_TEMP_2144" : {
"message" : [
"Unable to find constructor for <tpe>. This could happen if <tpe> is an interface, or a trait without companion object constructor."
Expand Down Expand Up @@ -6920,11 +6898,6 @@
"<clazz>: <msg>"
]
},
"_LEGACY_ERROR_TEMP_3066" : {
"message" : [
"<msg>"
]
},
"_LEGACY_ERROR_TEMP_3067" : {
"message" : [
"Streaming aggregation doesn't support group aggregate pandas UDF"
Expand Down Expand Up @@ -6980,11 +6953,6 @@
"More than one event time columns are available. Please ensure there is at most one event time column per stream. event time columns: <eventTimeCols>"
]
},
"_LEGACY_ERROR_TEMP_3078" : {
"message" : [
"Can not match ParquetTable in the query."
]
},
"_LEGACY_ERROR_TEMP_3079" : {
"message" : [
"Dynamic partition cannot be the parent of a static partition."
Expand Down
4 changes: 0 additions & 4 deletions docs/sql-error-conditions-invalid-handle-error-class.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,6 @@ Operation already exists.

Operation not found.

## SESSION_ALREADY_EXISTS

Session already exists.

## SESSION_CLOSED

Session was closed.
Expand Down
12 changes: 0 additions & 12 deletions docs/sql-error-conditions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -474,12 +474,6 @@ For more details see [DATATYPE_MISMATCH](sql-error-conditions-datatype-mismatch-

DataType `<type>` requires a length parameter, for example `<type>`(10). Please specify the length.

### DATA_SOURCE_ALREADY_EXISTS

[SQLSTATE: 42710](sql-error-conditions-sqlstates.html#class-42-syntax-error-or-access-rule-violation)

Data source '`<provider>`' already exists in the registry. Please use a different name for the new data source.

### DATA_SOURCE_NOT_EXIST

[SQLSTATE: 42704](sql-error-conditions-sqlstates.html#class-42-syntax-error-or-access-rule-violation)
Expand Down Expand Up @@ -886,12 +880,6 @@ You may get a different result due to the upgrading to

For more details see [INCONSISTENT_BEHAVIOR_CROSS_VERSION](sql-error-conditions-inconsistent-behavior-cross-version-error-class.html)

### INCORRECT_END_OFFSET

[SQLSTATE: 22003](sql-error-conditions-sqlstates.html#class-22-data-exception)

Max offset with `<rowsPerSecond>` rowsPerSecond is `<maxSeconds>`, but it's `<endSeconds>` now.

### INCORRECT_RAMP_UP_RATE

[SQLSTATE: 22003](sql-error-conditions-sqlstates.html#class-22-data-exception)
Expand Down
2 changes: 1 addition & 1 deletion sql/api/src/main/scala/org/apache/spark/sql/errors/DataTypeErrors.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import org.apache.spark.unsafe.types.UTF8String
/**
* Object for grouping error messages from (most) exceptions thrown during query execution.
* This does not include exceptions thrown during the eager execution of commands, which are
* grouped into [[QueryCompilationErrors]].
Copy link
Contributor Author

@panbingkun panbingkun Dec 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the sql/api module, QueryCompilationErrors has been renamed to CompilationErrors`, so we will fix it here.

Copy link
Member

@MaxGekk MaxGekk Dec 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have both, AFAIK

Copy link
Contributor Author

@panbingkun panbingkun Dec 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the perspective of module dependency, it seems to express the meaning of CompilationErrors in the sql/api module, instead of QueryCompilationErrors in the sql/catalyst module.

Or we can write it as: [[CompilationErrors]] and [[QueryCompilationErrors]] ?

Copy link
Contributor Author

@panbingkun panbingkun Dec 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The most primitive PR is here: https://github.com/apache/spark/pull/41794/files#diff-1b8affb347590e3ccabe359b07215c72aead8108ca3a5a020b3cb71e639c117bR25

Copy link
Member

@MaxGekk MaxGekk Dec 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, let's leave the ref to CompilationErrors only.

* grouped into [[CompilationErrors]].
*/
private[sql] object DataTypeErrors extends DataTypeErrorsBase {
def unsupportedOperationExceptionError(): SparkUnsupportedOperationException = {
Expand Down
6 changes: 0 additions & 6 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3856,12 +3856,6 @@ private[sql] object QueryCompilationErrors extends QueryErrorsBase with Compilat
"reason" -> reason))
}

def dataSourceAlreadyExists(name: String): Throwable = {
new AnalysisException(
errorClass = "DATA_SOURCE_ALREADY_EXISTS",
messageParameters = Map("provider" -> name))
}

def dataSourceDoesNotExist(name: String): Throwable = {
new AnalysisException(
errorClass = "DATA_SOURCE_NOT_EXIST",
Expand Down