Commit 93a09ea
committed
[SPARK-47072][SQL][3.5] Fix supported interval formats in error messages
### What changes were proposed in this pull request?
In the PR, I propose to add one more field to keys of `supportedFormat` in `IntervalUtils` because current implementation has duplicate keys that overwrites each other. For instance, the following keys are the same:
```
(YM.YEAR, YM.MONTH)
...
(DT.DAY, DT.HOUR)
```
because `YM.YEAR = DT.DAY = 0` and `YM.MONTH = DT.HOUR = 1`.
This is a backport of #45127.
### Why are the changes needed?
To fix the incorrect error message when Spark cannot parse ANSI interval string. For example, the expected format should be some year-month format but Spark outputs day-time one:
```sql
spark-sql (default)> select interval '-\t2-2\t' year to month;
Interval string does not match year-month format of `[+|-]d h`, `INTERVAL [+|-]'[+|-]d h' DAY TO HOUR` when cast to interval year to month: - 2-2 . (line 1, pos 16)
== SQL ==
select interval '-\t2-2\t' year to month
----------------^^^
```
### Does this PR introduce _any_ user-facing change?
Yes.
### How was this patch tested?
By running the existing test suite:
```
$ build/sbt "test:testOnly *IntervalUtilsSuite"
```
and regenerating the golden files:
```
$ SPARK_GENERATE_GOLDEN_FILES=1 PYSPARK_PYTHON=python3 build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite"
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Authored-by: Max Gekk <max.gekkgmail.com>
(cherry picked from commit 074fcf2)
Closes #45139 from MaxGekk/fix-supportedFormat-3.5.
Authored-by: Max Gekk <[email protected]>
Signed-off-by: Max Gekk <[email protected]>1 parent 5067447 commit 93a09ea
File tree
6 files changed
+27
-22
lines changed- sql
- catalyst/src
- main/scala/org/apache/spark/sql/catalyst/util
- test/scala/org/apache/spark/sql/catalyst/expressions
- core/src/test/resources/sql-tests
- analyzer-results
- ansi
- results
- ansi
6 files changed
+27
-22
lines changedLines changed: 19 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
107 | 107 | | |
108 | 108 | | |
109 | 109 | | |
110 | | - | |
| 110 | + | |
| 111 | + | |
111 | 112 | | |
112 | 113 | | |
113 | 114 | | |
114 | 115 | | |
115 | 116 | | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
120 | | - | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
127 | | - | |
128 | | - | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
129 | 134 | | |
130 | 135 | | |
131 | 136 | | |
| |||
Lines changed: 4 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1174 | 1174 | | |
1175 | 1175 | | |
1176 | 1176 | | |
1177 | | - | |
| 1177 | + | |
1178 | 1178 | | |
1179 | 1179 | | |
1180 | 1180 | | |
| |||
1194 | 1194 | | |
1195 | 1195 | | |
1196 | 1196 | | |
1197 | | - | |
| 1197 | + | |
1198 | 1198 | | |
1199 | 1199 | | |
1200 | 1200 | | |
| |||
1314 | 1314 | | |
1315 | 1315 | | |
1316 | 1316 | | |
1317 | | - | |
| 1317 | + | |
1318 | 1318 | | |
1319 | 1319 | | |
1320 | 1320 | | |
| |||
1338 | 1338 | | |
1339 | 1339 | | |
1340 | 1340 | | |
1341 | | - | |
| 1341 | + | |
1342 | 1342 | | |
1343 | 1343 | | |
1344 | 1344 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1916 | 1916 | | |
1917 | 1917 | | |
1918 | 1918 | | |
1919 | | - | |
| 1919 | + | |
1920 | 1920 | | |
1921 | 1921 | | |
1922 | 1922 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1916 | 1916 | | |
1917 | 1917 | | |
1918 | 1918 | | |
1919 | | - | |
| 1919 | + | |
1920 | 1920 | | |
1921 | 1921 | | |
1922 | 1922 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2355 | 2355 | | |
2356 | 2356 | | |
2357 | 2357 | | |
2358 | | - | |
| 2358 | + | |
2359 | 2359 | | |
2360 | 2360 | | |
2361 | 2361 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2168 | 2168 | | |
2169 | 2169 | | |
2170 | 2170 | | |
2171 | | - | |
| 2171 | + | |
2172 | 2172 | | |
2173 | 2173 | | |
2174 | 2174 | | |
| |||
0 commit comments