Commit 73da9c2
[SPARK-3771][SQL] AppendingParquetOutputFormat should use reflection to prevent from breaking binary-compatibility.
Original problem is [SPARK-3764](https://issues.apache.org/jira/browse/SPARK-3764).
`AppendingParquetOutputFormat` uses a binary-incompatible method `context.getTaskAttemptID`.
This causes binary-incompatible of Spark itself, i.e. if Spark itself is built against hadoop-1, the artifact is for only hadoop-1, and vice versa.
Author: Takuya UESHIN <[email protected]>
Closes #2638 from ueshin/issues/SPARK-3771 and squashes the following commits:
efd3784 [Takuya UESHIN] Add a comment to explain the reason to use reflection.
ec213c1 [Takuya UESHIN] Use reflection to prevent breaking binary-compatibility.1 parent d3cdf91 commit 73da9c2
File tree
1 file changed
+9
-1
lines changed- sql/core/src/main/scala/org/apache/spark/sql/parquet
1 file changed
+9
-1
lines changedLines changed: 9 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
331 | 331 | | |
332 | 332 | | |
333 | 333 | | |
334 | | - | |
| 334 | + | |
335 | 335 | | |
336 | 336 | | |
337 | 337 | | |
338 | 338 | | |
339 | 339 | | |
340 | 340 | | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
341 | 349 | | |
342 | 350 | | |
343 | 351 | | |
| |||
0 commit comments