-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-40400][SQL] Pass error message parameters to exceptions as maps #37834
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@cloud-fan @gengliangwang Could you review this PR, please. |
| with SparkThrowable { | ||
|
|
||
| override def getMessageParameters: Array[String] = { | ||
| SparkThrowableHelper.getMessageParameters(errorClass, errorSubClass.orNull, messageParameters) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1. These 2 methods were added by #36693 and not released yet. We can still change it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree. Let's do that in a separate PR.
|
Merging to master. Thank you, @srielau @gengliangwang @cloud-fan for review. |
| "proposal" -> | ||
| "`__auto_generated_subquery_name`.`c1`, `__auto_generated_subquery_name`.`c2`"), | ||
| context = ExpectedContext( | ||
| fragment = insert, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MaxGekk is it a bug? The error context should be c3 in this case, instead of the INSERT statement.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it should be c3. The PR #37861 changes this in OSS.
What changes were proposed in this pull request?
In the PR, I propose to change the type of
messageParameterspassed to Spark exception (via constructors) from array of strings to map of pair of strings. The map contains parameters names fromerror-classes.jsonas keys, and parameters values as map's values. While formatting error messages using the templates fromerror-classes.json, the formatter can fail w/ an internal error if it cannot find a parameter in the map passed to the current exception. The map of message parameters can contain more elements then used by the template.Before:
After:
Why are the changes needed?
At the moment, the order of error message parameters cannot be changed in
error-classes.jsonindependently from the source code where the parameters are passed as arrays. After the changes, the tech editors/users can change the error messages including the order of params or skip some params w/ modifying Spark's code base (including tests).Does this PR introduce any user-facing change?
Yes.
How was this patch tested?
By running the modified test suites: