Commit f7ba952
[SPARK-33048][BUILD] Fix SparkBuild.scala to recognize build settings for Scala 2.13
### What changes were proposed in this pull request?
This PR fixes `SparkBuild.scala` to recognize build settings for Scala 2.13.
In `SparkBuild.scala`, a variable `scalaBinaryVersion` is hardcoded as `2.12`.
So, an environment variable `SPARK_SCALA_VERSION` is also to be `2.12`.
This issue causes some test suites (e.g. `SparkSubmitSuite`) to be error.
```
===== TEST OUTPUT FOR o.a.s.deploy.SparkSubmitSuite: 'user classpath first in driver' =====
20/10/02 08:55:30.234 redirect stderr for command /home/kou/work/oss/spark-scala-2.13/bin/spark-submit INFO Utils: Error: Could not find or load m
ain class org.apache.spark.launcher.Main
20/10/02 08:55:30.235 redirect stderr for command /home/kou/work/oss/spark-scala-2.13/bin/spark-submit INFO Utils: /home/kou/work/oss/spark-scala-
2.13/bin/spark-class: line 96: CMD: bad array subscript
```
The reason of this error is that environment variables `SPARK_JARS_DIR` and `LAUNCH_CLASSPATH` is defined in `bin/spark-class` as follows.
```
SPARK_JARS_DIR="${SPARK_HOME}/assembly/target/scala-$SPARK_SCALA_VERSION/jars"
LAUNCH_CLASSPATH="${SPARK_HOME}/launcher/target/scala-$SPARK_SCALA_VERSION/classes:$LAUNCH_CLASSPATH"
```
### Why are the changes needed?
To build for Scala 2.13 successfully.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Tests for `core` module finish successfully.
```
build/sbt -Pscala-2.13 clean "core/test"
```
Closes #29927 from sarutak/fix-sparkbuild-for-scala-2.13.
Authored-by: Kousuke Saruta <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>1 parent b205be5 commit f7ba952
1 file changed
+1
-27
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
94 | 94 | | |
95 | 95 | | |
96 | 96 | | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
109 | | - | |
110 | | - | |
111 | | - | |
112 | 97 | | |
113 | 98 | | |
114 | 99 | | |
| |||
965 | 950 | | |
966 | 951 | | |
967 | 952 | | |
968 | | - | |
969 | | - | |
970 | | - | |
971 | | - | |
972 | | - | |
973 | | - | |
974 | | - | |
975 | | - | |
976 | | - | |
977 | | - | |
978 | | - | |
979 | 953 | | |
980 | 954 | | |
981 | 955 | | |
| |||
988 | 962 | | |
989 | 963 | | |
990 | 964 | | |
991 | | - | |
| 965 | + | |
992 | 966 | | |
993 | 967 | | |
994 | 968 | | |
| |||
0 commit comments