Commit 9b0971c
[SPARK-35990][BUILD] Remove avro-sbt plugin dependency
### What changes were proposed in this pull request?
This PR removes sbt-avro plugin dependency.
In the current master, Build with SBT depends on the plugin but it seems never used.
Originally, the plugin was introduced for `flume-sink` in SPARK-1729 (#807) but `flume-sink` is no longer in Spark repository.
After SBT was upgraded to 1.x in SPARK-21708 (#29286), `avroGenerate` part was introduced in `object SQL` in `SparkBuild.scala`.
It's confusable but I understand `Test / avroGenerate := (Compile / avroGenerate).value` is for suppressing sbt-avro for `sql` sub-module.
In fact, Test/compile will fail if `Test / avroGenerate :=(Compile / avroGenerate).value` is commented out.
`sql` sub-module contains `parquet-compat.avpr` and `parquet-compat.avdl` but according to `sql/core/src/test/README.md`, they are intended to be handled by `gen-avro.sh`.
Also, in terms of Maven build, there seems to be no definition to handle `*.avpr` or `*.avdl`.
Based on the above, I think we can remove `sbt-avro`.
### Why are the changes needed?
If `sbt-avro` is really no longer used, it's confusable that `sbt-avro` related configurations are in `SparkBuild.scala`.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
GA.
Closes #33190 from sarutak/remove-avro-from-sbt.
Authored-by: Kousuke Saruta <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 6c4616b)
Signed-off-by: Dongjoon Hyun <[email protected]>1 parent 0db6f3a commit 9b0971c
2 files changed
+1
-9
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
737 | 737 | | |
738 | 738 | | |
739 | 739 | | |
740 | | - | |
741 | | - | |
742 | | - | |
743 | 740 | | |
744 | 741 | | |
745 | 742 | | |
| |||
761 | 758 | | |
762 | 759 | | |
763 | 760 | | |
764 | | - | |
765 | | - | |
| 761 | + | |
766 | 762 | | |
767 | | - | |
768 | 763 | | |
769 | 764 | | |
770 | 765 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | 36 | | |
40 | 37 | | |
41 | 38 | | |
| |||
0 commit comments