[SPARK-39497][SQL] Improve the analysis exception of missing map key column #36896
Conversation
|
cc @srielau as well |
| } | ||
|
|
||
| // If an attribute can't be resolved as a map key of string type, either the key should be | ||
| // surrounded with single quotes, or there is a typo in the attribute name. |
There was a problem hiding this comment.
I tried to match element_at and try_element_at here. But before the functions are not resolved yet in the rule ResolveFunctions, the error MISSING_COLUMN is thrown.
Checking the unresolved function name and arguments can work but is quite ugly. I decided to handle the [] operator first.
| checkAnswer(sql("select m['a'] from (select map('a', 'b') as m, 'aa' as aa)"), Row("b")) | ||
| checkErrorClass( | ||
| exception = intercept[AnalysisException] { | ||
| sql("select m[a] from (select map('a', 'b') as m, 'aa' as aa)") |
There was a problem hiding this comment.
In CheckAnalysis, how can we distinguish m.a and m[a]?
There was a problem hiding this comment.
m.a is equivalent to m['a'] so it won't fail analysis
| val orderedCandidates = StringUtils.orderStringsBySimilarity(unresolvedAttribute, candidates) | ||
| a.failAnalysis( | ||
| errorClass = errorClass, | ||
| messageParameters = Array(unresolvedAttribute, orderedCandidates.mkString(", "))) |
There was a problem hiding this comment.
Both candidates and unresolvedAttribute should be wrapped with toSQLId
There was a problem hiding this comment.
I see. I will update this one after #36891 is merged.
srielau
left a comment
srielau
left a comment
There was a problem hiding this comment.
Added a comment about toSQLId
|
Merging to master |
3 participants
What changes were proposed in this pull request?
Sometimes users forgot to add single quotes on the map key string literal, for example
map_col[a]. In such a case, the Analyzer will throw an exception:We can improve this message by saying that the user should append single quotes if the map key is a string literal.
Why are the changes needed?
Error message improvement
Does this PR introduce any user-facing change?
Yes but trivial, an improvement on the error message of unresolved map key column
How was this patch tested?
New UT