[SPARK-49516] Upgrade the minimum K8s version to v1.28

### What changes were proposed in this pull request?

This PR aims to upgrade the minimum K8s version to v1.28.
- This is aligned with apache/spark#47990 .

In addition, this PR renames the test matrix names to use `hypen` instead of `underscore`.
- `kubernetes_version` -> `kubernetes-version`
- `test_group` -> `test-group`
- `dynamic_config_test_group` -> `dynamic-config-test-group`

### Why are the changes needed?

**1. K8s community archived v1.27.19 on 2024-07-16 and starts to release v1.31.0 from 2024-08-13**
- https://kubernetes.io/releases/#release-v1-31
- https://kubernetes.io/releases/patch-releases/#non-active-branch-history

**2. Default K8s Versions in Public Cloud environments**

The default K8s versions of public cloud providers are already moving to K8s 1.30 like the following.

- EKS: v1.30 (Default)
- GKE: v1.30 (Rapid),  v1.29 (Regular), v1.29 (Stable)
- AKS: v1.29 (Default), v1.30 (Support)

**3. End Of Support**

In addition, K8s 1.27 reached or will reach a standard support EOL in two weeks before the official Apache Spark Kubernetes Operator release.

| K8s  |   EKS   |  AKS  |   GKE   |
| ---- | ------- | ------- | ------- |
| 1.27 | 2024-07 | 2024-07 | 2024-09-16 |

- [EKS EOL Schedule](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html#kubernetes-release-calendar)
- [AKS EOL Schedule](https://docs.microsoft.com/en-us/azure/aks/supported-kubernetes-versions?tabs=azure-cli#aks-kubernetes-release-calendar)
- [GKE EOL Schedule](https://cloud.google.com/kubernetes-engine/docs/release-schedule)

### Does this PR introduce _any_ user-facing change?

No. This is a test infra change.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #117 from dongjoon-hyun/SPARK-49516.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>

Author: dongjoon-hyun

.github/workflows/build_and_test.yml

@@ -66,14 +66,14 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        kubernetes_version:
-          - "1.27.0"
+        kubernetes-version:
+          - "1.28.0"
           - "1.31.0"
-        test_group:
+        test-group:
           - spark-versions
           - python
           - state-transition
-        dynamic_config_test_group:
+        dynamic-config-test-group:
           - watched-namespaces
     steps:
       - name: Checkout repository
@@ -88,7 +88,7 @@ jobs:
         uses: medyagh/[email protected]
         with:
           cache: true
-          kubernetes-version: ${{ matrix.kubernetes_version }}
+          kubernetes-version: ${{ matrix.kubernetes-version }}
           cpus: 2
           memory: 6144m
       - name: Set Up Go
@@ -112,7 +112,7 @@ jobs:
           minikube docker-env --unset
       - name: Run E2E Test with Dynamic Configuration Disabled
         run: |
-          chainsaw test --test-dir ./tests/e2e/${{ matrix.test_group }} --parallel 2
+          chainsaw test --test-dir ./tests/e2e/${{ matrix.test-group }} --parallel 2
       - name: Run Spark K8s Operator on K8S with Dynamic Configuration Enabled
         run: |
           helm uninstall spark-kubernetes-operator
@@ -124,7 +124,7 @@ jobs:
           minikube docker-env --unset
       - name: Run E2E Test with Dynamic Configuration Enabled
         run: |
-          chainsaw test --test-dir ./tests/e2e/${{ matrix.dynamic_config_test_group }} --parallel 2
+          chainsaw test --test-dir ./tests/e2e/${{ matrix.dynamic-config-test-group }} --parallel 2
 
   lint:
     name: "Linter and documentation"