Run e2e tests with Kubernetes version matrix

[jacobsalway]

Purpose of this PR

• Run e2e tests with Kubernetes version matrix containing n-3 of the current Kubernetes versions

Change Category

Indicate the type of change by marking the applicable boxes:

• Bugfix (non-breaking change which fixes an issue)
• Feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that could affect existing functionality)
• Documentation update

Checklist

• I have conducted a self-review of my own code.
• I have updated documentation accordingly.
• I have added tests that prove my changes are effective or that my feature works.
• Existing unit tests pass locally with my changes.

[jacobsalway]

/assign @ChenYi015 @ImpSy

[ChenYi015]

Maybe we should change default k8s version to v1.29.3 as we use now.

[jacobsalway]

Do we need to keep ENVTEST_K8S_VERSION, KIND_K8S_VERSION and the Go client versions in sync? If there's no dependency I'd prefer to use the latest minor version i.e. 1.31

[ChenYi015]

Yes, I think we should keep the three k8s versions the same.

[jacobsalway]

Changed to 1.29.3. I reckon we could bump to the latest minor at some point soon though. I don't think there are any APIs that either Spark or the operator are using that would block us, unless there are any kubebuilder/controller-runtime dependency issues.

[jacobsalway]

I'll document the official n-3 (as governed by e2e test success) compatibility somewhere in a subsequent docs PR. I'd like to update the docs as mentioned during the community call.

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ChenYi015

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ChenYi015]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ChenYi015]

/lgtm