Add additional images to Apache Spark 3.5.0#15697
Merged
yosifkit merged 1 commit intodocker-library:masterfrom Nov 16, 2023
Merged
Add additional images to Apache Spark 3.5.0#15697yosifkit merged 1 commit intodocker-library:masterfrom
yosifkit merged 1 commit intodocker-library:masterfrom
Conversation
This patch adds additional images to 3.5.0 version for Apache Spark https://spark.apache.org/docs/3.5.0/ [1] [1] apache/spark-website#476 [2] apache/spark-docker#55
Diff for 6094bd3:diff --git a/_bashbrew-cat b/_bashbrew-cat
index 71edde5..cd0b29f 100644
--- a/_bashbrew-cat
+++ b/_bashbrew-cat
@@ -40,3 +40,23 @@ Tags: 3.5.0-scala2.12-java11-ubuntu, 3.5.0-scala, scala
Architectures: amd64, arm64v8
GitCommit: 028efd4637fb2cf791d5bd9ea70b2fca472de4b7
Directory: 3.5.0/scala2.12-java11-ubuntu
+
+Tags: 3.5.0-scala2.12-java17-python3-r-ubuntu
+Architectures: amd64, arm64v8
+GitCommit: 6f68fe0f7051c10f2bf43a50a7decfce2e97baf0
+Directory: 3.5.0/scala2.12-java17-python3-r-ubuntu
+
+Tags: 3.5.0-scala2.12-java17-python3-ubuntu, 3.5.0-java17-python3, 3.5.0-java17, python3-java17
+Architectures: amd64, arm64v8
+GitCommit: 6f68fe0f7051c10f2bf43a50a7decfce2e97baf0
+Directory: 3.5.0/scala2.12-java17-python3-ubuntu
+
+Tags: 3.5.0-scala2.12-java17-r-ubuntu, 3.5.0-java17-r
+Architectures: amd64, arm64v8
+GitCommit: 6f68fe0f7051c10f2bf43a50a7decfce2e97baf0
+Directory: 3.5.0/scala2.12-java17-r-ubuntu
+
+Tags: 3.5.0-scala2.12-java17-ubuntu, 3.5.0-java17-scala
+Architectures: amd64, arm64v8
+GitCommit: 6f68fe0f7051c10f2bf43a50a7decfce2e97baf0
+Directory: 3.5.0/scala2.12-java17-ubuntu
diff --git a/_bashbrew-list b/_bashbrew-list
index 19035e4..f93ec40 100644
--- a/_bashbrew-list
+++ b/_bashbrew-list
@@ -7,6 +7,10 @@ spark:3.4.1-scala2.12-java11-python3-ubuntu
spark:3.4.1-scala2.12-java11-r-ubuntu
spark:3.4.1-scala2.12-java11-ubuntu
spark:3.5.0
+spark:3.5.0-java17
+spark:3.5.0-java17-python3
+spark:3.5.0-java17-r
+spark:3.5.0-java17-scala
spark:3.5.0-python3
spark:3.5.0-r
spark:3.5.0-scala
@@ -14,7 +18,12 @@ spark:3.5.0-scala2.12-java11-python3-r-ubuntu
spark:3.5.0-scala2.12-java11-python3-ubuntu
spark:3.5.0-scala2.12-java11-r-ubuntu
spark:3.5.0-scala2.12-java11-ubuntu
+spark:3.5.0-scala2.12-java17-python3-r-ubuntu
+spark:3.5.0-scala2.12-java17-python3-ubuntu
+spark:3.5.0-scala2.12-java17-r-ubuntu
+spark:3.5.0-scala2.12-java17-ubuntu
spark:latest
spark:python3
+spark:python3-java17
spark:r
spark:scala
diff --git a/_bashbrew-list-build-order b/_bashbrew-list-build-order
index 168fcfe..b9203b3 100644
--- a/_bashbrew-list-build-order
+++ b/_bashbrew-list-build-order
@@ -1,8 +1,12 @@
spark:3.4.1-scala
spark:scala
+spark:3.5.0-java17-scala
spark:3.4.1-scala2.12-java11-python3-r-ubuntu
spark:3.4.1
spark:3.4.1-r
spark:3.5.0-scala2.12-java11-python3-r-ubuntu
spark:latest
spark:r
+spark:3.5.0-scala2.12-java17-python3-r-ubuntu
+spark:python3-java17
+spark:3.5.0-java17-r
diff --git a/spark_3.4.1-r/Dockerfile b/spark_3.5.0-java17-r/Dockerfile
similarity index 95%
copy from spark_3.4.1-r/Dockerfile
copy to spark_3.5.0-java17-r/Dockerfile
index 1c9fc38..753f3cf 100644
--- a/spark_3.4.1-r/Dockerfile
+++ b/spark_3.5.0-java17-r/Dockerfile
@@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-FROM spark:3.4.1-scala2.12-java11-ubuntu
+FROM spark:3.5.0-scala2.12-java17-ubuntu
USER root
diff --git a/spark_scala/Dockerfile b/spark_3.5.0-java17-scala/Dockerfile
similarity index 98%
copy from spark_scala/Dockerfile
copy to spark_3.5.0-java17-scala/Dockerfile
index 15f4b31..ed29cba 100644
--- a/spark_scala/Dockerfile
+++ b/spark_3.5.0-java17-scala/Dockerfile
@@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-FROM eclipse-temurin:11-jre-focal
+FROM eclipse-temurin:17-jre-jammy
ARG spark_uid=185
diff --git a/spark_scala/entrypoint.sh b/spark_3.5.0-java17-scala/entrypoint.sh
similarity index 100%
copy from spark_scala/entrypoint.sh
copy to spark_3.5.0-java17-scala/entrypoint.sh
diff --git a/spark_3.4.1-scala2.12-java11-python3-r-ubuntu/Dockerfile b/spark_3.5.0-scala2.12-java17-python3-r-ubuntu/Dockerfile
similarity index 96%
copy from spark_3.4.1-scala2.12-java11-python3-r-ubuntu/Dockerfile
copy to spark_3.5.0-scala2.12-java17-python3-r-ubuntu/Dockerfile
index 30e6b86..fd05497 100644
--- a/spark_3.4.1-scala2.12-java11-python3-r-ubuntu/Dockerfile
+++ b/spark_3.5.0-scala2.12-java17-python3-r-ubuntu/Dockerfile
@@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-FROM spark:3.4.1-scala2.12-java11-ubuntu
+FROM spark:3.5.0-scala2.12-java17-ubuntu
USER root
diff --git a/spark_latest/Dockerfile b/spark_python3-java17/Dockerfile
similarity index 95%
copy from spark_latest/Dockerfile
copy to spark_python3-java17/Dockerfile
index 78db218..e57b4f5 100644
--- a/spark_latest/Dockerfile
+++ b/spark_python3-java17/Dockerfile
@@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-FROM spark:3.5.0-scala2.12-java11-ubuntu
+FROM spark:3.5.0-scala2.12-java17-ubuntu
USER rootRelevant Maintainers:
|
yosifkit
approved these changes
Nov 14, 2023
Member
|
This looks correct to me and sanctioned by the maintainers: apache/spark-docker#56 (comment). @Yikun, let me know it this looks good to you. |
Contributor
|
@yosifkit Thanks! Consider it's the first java 17 series images, I sent a mail to spark dev mail list to see any other concerns from Spark community. I will back here before Friday to give the final +1, Thanks! [1] https://lists.apache.org/thread/ryycljltozq25j0o5mn9qbwypqso4k5p also cc @HyukjinKwon @zhengruifeng |
HyukjinKwon
reviewed
Nov 14, 2023
Contributor
HyukjinKwon
left a comment
HyukjinKwon
left a comment
There was a problem hiding this comment.
I am good with this +1
Contributor
|
I'm also good with this! +1 No -1 from Spark community, https://lists.apache.org/thread/ryycljltozq25j0o5mn9qbwypqso4k5p @yosifkit Pls feel free to merge! Much thanks for your help! @HyukjinKwon @vakarisbk Thanks all! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This patch adds additional images to 3.5.0 version of Apache Spark https://spark.apache.org/docs/3.5.0/
apache/spark-docker#56