Commit 074fcf2
committed
[SPARK-47072][SQL] Fix supported interval formats in error messages
### What changes were proposed in this pull request?
In the PR, I propose to add one more field to keys of `supportedFormat` in `IntervalUtils` because current implementation has duplicate keys that overwrites each other. For instance, the following keys are the same:
```
(YM.YEAR, YM.MONTH)
...
(DT.DAY, DT.HOUR)
```
because `YM.YEAR = DT.DAY = 0` and `YM.MONTH = DT.HOUR = 1`.
### Why are the changes needed?
To fix the incorrect error message when Spark cannot parse ANSI interval string. For example, the expected format should be some year-month format but Spark outputs day-time one:
```sql
spark-sql (default)> select interval '-\t2-2\t' year to month;
Interval string does not match year-month format of `[+|-]d h`, `INTERVAL [+|-]'[+|-]d h' DAY TO HOUR` when cast to interval year to month: - 2-2 . (line 1, pos 16)
== SQL ==
select interval '-\t2-2\t' year to month
----------------^^^
```
### Does this PR introduce _any_ user-facing change?
Yes.
### How was this patch tested?
By running the existing test suite:
```
$ build/sbt "test:testOnly *IntervalUtilsSuite"
```
and regenerating the golden files:
```
$ SPARK_GENERATE_GOLDEN_FILES=1 PYSPARK_PYTHON=python3 build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite"
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes apache#45127 from MaxGekk/fix-supportedFormat.
Authored-by: Max Gekk <[email protected]>
Signed-off-by: Max Gekk <[email protected]>1 parent 64fa13b commit 074fcf2
File tree
6 files changed
+26
-22
lines changed- sql
- catalyst/src
- main/scala/org/apache/spark/sql/catalyst/util
- test/scala/org/apache/spark/sql/catalyst/expressions
- core/src/test/resources/sql-tests
- analyzer-results
- ansi
- results
- ansi
6 files changed
+26
-22
lines changedLines changed: 18 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
110 | 110 | | |
111 | 111 | | |
112 | 112 | | |
113 | | - | |
| 113 | + | |
114 | 114 | | |
115 | 115 | | |
116 | 116 | | |
117 | 117 | | |
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
134 | 138 | | |
135 | 139 | | |
136 | 140 | | |
| |||
Lines changed: 4 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1181 | 1181 | | |
1182 | 1182 | | |
1183 | 1183 | | |
1184 | | - | |
| 1184 | + | |
1185 | 1185 | | |
1186 | 1186 | | |
1187 | 1187 | | |
| |||
1204 | 1204 | | |
1205 | 1205 | | |
1206 | 1206 | | |
1207 | | - | |
| 1207 | + | |
1208 | 1208 | | |
1209 | 1209 | | |
1210 | 1210 | | |
| |||
1329 | 1329 | | |
1330 | 1330 | | |
1331 | 1331 | | |
1332 | | - | |
| 1332 | + | |
1333 | 1333 | | |
1334 | 1334 | | |
1335 | 1335 | | |
| |||
1355 | 1355 | | |
1356 | 1356 | | |
1357 | 1357 | | |
1358 | | - | |
| 1358 | + | |
1359 | 1359 | | |
1360 | 1360 | | |
1361 | 1361 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1916 | 1916 | | |
1917 | 1917 | | |
1918 | 1918 | | |
1919 | | - | |
| 1919 | + | |
1920 | 1920 | | |
1921 | 1921 | | |
1922 | 1922 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1916 | 1916 | | |
1917 | 1917 | | |
1918 | 1918 | | |
1919 | | - | |
| 1919 | + | |
1920 | 1920 | | |
1921 | 1921 | | |
1922 | 1922 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2355 | 2355 | | |
2356 | 2356 | | |
2357 | 2357 | | |
2358 | | - | |
| 2358 | + | |
2359 | 2359 | | |
2360 | 2360 | | |
2361 | 2361 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2168 | 2168 | | |
2169 | 2169 | | |
2170 | 2170 | | |
2171 | | - | |
| 2171 | + | |
2172 | 2172 | | |
2173 | 2173 | | |
2174 | 2174 | | |
| |||
0 commit comments