ci-build-and-push-k8s-at-golang-tip should test tip of k/k and k/release

[dims]

Let's drop the hard coded SHA(s) please

xref: #34237

[dims]

/assign @liggitt @BenTheElder

[BenTheElder]

/lgtm
/approve

[BenTheElder]

Also: We should verify that this is still actually testing with go tip, k/k will control the go version unless you override that and I don't see an override here

[BenTheElder]

/hold

[dims]

@BenTheElder i want to see what happens and then will start tweaking it again

[BenTheElder]

does this repo's makefile handle clobbering GOTOOLCHAIN? if not we should add that somewhere

[BenTheElder]

I don't think so, but let's patch it there https://github.com/kubernetes/perf-tests/blob/master/golang/

[BenTheElder]

kubernetes/perf-tests#3146

[tosi3k]

/hold

The intention of this test, together with ci-golang-tip-k8s-1-23, was to detect performance regressions in K8s that would come from the new Golang versions, not the K8s code itself. This test builds a fixed K8s version using newest Golang version and pushes it to some GCS bucket and ci-golang-tip-k8s-1-23 uses that version to check whether or not newest Golang version contains some performance regression that might be detrimental to K8s's performance.

This is the reason why we use fixed commits for the k/k repo in this job - to make sure that we are able to narrow down such regressions before they are even released in Golang itself and not get noise from the changes to the K8s code in case of a test failure. The k/release repo is also fixed because changes in the scripts from that repo that are used in this CI job had negative impact on that test's stability.

Before we had those CI jobs, it took us weeks to narrow down the performance regressions in K8s coming from Golang. These CI jobs allow us to catch them much more quickly, even before they reach Go releases.

Rather than using master branch directly for the k8s.io/{kubernetes,release} repos in that test, I'd suggest changing ci-golang-tip-k8s-1-23 in the following way:

1. Make it use either --extract=gs://k8s-infra-scale-golang-builds/ci/latest.txt or --extract=gs://k8s-infra-scale-golang-builds/ci/latest-1.32.txt.
   • The K8s version used in the ci-build-and-push-k8s-at-golang-tip CI job was bumped recently to 1.32 in Bump revision of commit used to verify latest golang compiler #33436.
   • Apparently, that PR forgot to update the --extract flag accordingly - thanks for catching that.
2. Rename that test to ci-golang-tip-k8s-1-32.

[BenTheElder]

The problem with that approach is that it's not staying up to date, we should use the latest stable tagged release or something and stop hardcoding specific versions

The K8s version used in the ci-build-and-push-k8s-at-golang-tip CI job was bumped recently to 1.32 in #33436.

Yes... before which it sat for almost three years exactly meaning it was on an unsupported branch for O(years)

-    base_sha: 0ff1922aa2269879957646a232f657bcb57824b1 # head of master branch as of 2021-09-25
+    base_sha: 75a8e7ef8b22fe41525d28c1818384aab795f145 # head of master branch as of 2024-09-05

Can we use https://github.com/kubernetes/community/blob/322066e7dba7c5043071392fec427a57f8660734/contributors/devel/sig-release/kubernetes-versions.md to get k8s-stable1 isntead?
cc @kubernetes/release-engineering

[BenTheElder]

BTW You should check these binaries, I don't think it's actually using the intended go version anyhow for years now: kubernetes/perf-tests#3146

AFAICT this tooling is not overriding how go builds actually work in Kubernetes when it attempts to locally commit a patch to the k/k sources (!), not only is this fragile but at a glance the builds are ignoring go tip and using .go-version anyhow since we've been using GOTOOLCHAIN for some time now.

If this script had been using the kubernetes-specific GO_VERSION env prior to GOTOOLCHAIN we shimmed that, but the approach of patching the build scripts with a local commit doesn't work.

Also, this means the reported commit in the build is a commit that doesn't exist upstream.

kubernetes/perf-tests#3146

[saschagrunert]

LGTM

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenTheElder, dims, saschagrunert

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• config/jobs/kubernetes/sig-scalability/OWNERS [BenTheElder,dims]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment