[SPARK-49715] Add Java 21-based SparkCluster example

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to add Java 21-based SparkCluster example.

Why are the changes needed?

Apache Spark starts to publish Java 21-based image from Today (2024-09-19). This new example will illustrate how to use it.

• [SPARK-49703] Publish Java 21 Docker image for preview1 spark-docker#69

Does this PR introduce any user-facing change?

No, this is a new example.

How was this patch tested?

Manual review.

$ kubectl apply -f cluster-java21.yaml

$ kubectl get sparkcluster
NAME             CURRENT STATE    AGE
cluster-java21   RunningHealthy   9s

$ kubectl describe sparkcluster cluster-java21
Name:         cluster-java21
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  spark.apache.org/v1alpha1
Kind:         SparkCluster
Metadata:
  Creation Timestamp:  2024-09-19T04:25:30Z
  Finalizers:
    sparkclusters.spark.apache.org/finalizer
  Generation:        2
  Resource Version:  96663
  UID:               9421c957-380d-4a26-a266-379304bf83ee
Spec:
  Cluster Tolerations:
    Instance Config:
      Init Workers:  3
      Max Workers:   3
      Min Workers:   3
  Master Spec:
  Runtime Versions:
    Spark Version:  4.0.0-preview1
  Spark Conf:
    spark.kubernetes.container.image:  apache/spark:4.0.0-preview1-java21
    spark.master.rest.enabled:         true
    spark.master.rest.host:            0.0.0.0
    spark.master.ui.title:             Prod Spark Cluster (Java 21)
    spark.ui.reverseProxy:             true
  Worker Spec:
Status:
  Current Attempt Summary:
    Attempt Info:
      Id:  0
  Current State:
    Current State Summary:  RunningHealthy
    Last Transition Time:   2024-09-19T04:25:30.665095088Z
    Message:                Cluster has reached ready state.
  State Transition History:
    0:
      Current State Summary:  Submitted
      Last Transition Time:   2024-09-19T04:25:30.640072963Z
      Message:                Spark cluster has been submitted to Kubernetes Cluster.
    1:
      Current State Summary:  RunningHealthy
      Last Transition Time:   2024-09-19T04:25:30.665095088Z
      Message:                Cluster has reached ready state.
Events:                       <none>

$ k get pod
NAME                                         READY   STATUS    RESTARTS   AGE
cluster-java21-master-0                      1/1     Running   0          3m20s
cluster-java21-worker-0                      1/1     Running   0          3m20s
cluster-java21-worker-1                      1/1     Running   0          3m20s
cluster-java21-worker-2                      1/1     Running   0          3m20s
spark-kubernetes-operator-778b9bbdc6-fqks9   1/1     Running   0          20m

Was this patch authored or co-authored using generative AI tooling?

No.

[dongjoon-hyun]

cc @viirya

[dongjoon-hyun]

Thank you. Merged to main.