Use correct main branch in image publishing of external-snapshot-metadata

[hairyhum]

Follow-up to #33303

Addressing #33303 (comment)

[k8s-ci-robot]

Hi @hairyhum. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[hairyhum]

/assign @xing-yang @Rakshith-R

[k8s-ci-robot]

@hairyhum: GitHub didn't allow me to assign the following users: Rakshith-R.

Note that only kubernetes members with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @xing-yang @Rakshith-R

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[xing-yang]

/ok-to-test

[Rakshith-R]

There's one more place that needs the same change.

test-infra/config/jobs/image-pushing/k8s-staging-sig-storage.yaml

Line 343 in b072da3

- ^master$

[xing-yang]

Hmm, we have a problem here. This is an auto-generated file.

[xing-yang]

You'll need to modify gen-jobs.sh to automatically set it to "main" for external-snapshot-metadata: https://github.com/kubernetes/test-infra/pull/33303/files#diff-a8e1d6d57695368eeaaf2c7b80d75ce59078cea09d2218469a6a5dc7009dc8f5

[hairyhum]

@xing-yang @Rakshith-R updated the template and re-generated.

[xing-yang]

This will break jobs for all other sidecars.

[Rakshith-R]

test-infra/config/jobs/image-pushing/k8s-staging-sig-storage.sh

Lines 139 to 142 in 96376be

	# We need to emulate a pull job for the cloud build to work the same
	# way as it usually does.
	- name: PULL_BASE_REF
	value: master

This needs to be changed to main too for external-snapshot-metadata side car.

---

Maybe conditionally setting the default branch by checking the name of the repo works ?
@hairyhum Can you please try this approach ?

#!/usr/bin/env bash
# Copyright 2020 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit

readonly OUTPUT="$(dirname $0)/k8s-staging-sig-storage.yaml"
echo $OUPUT
# Repos for which cloud image builds are working.
readonly REPOS=(
    kubernetes-csi/csi-driver-host-path
    kubernetes-csi/csi-driver-smb
    kubernetes-csi/csi-test
    kubernetes-csi/external-attacher
    kubernetes-csi/external-health-monitor
    kubernetes-csi/external-provisioner
    kubernetes-csi/external-resizer
    kubernetes-csi/external-snapshotter
    kubernetes-csi/external-snapshot-metadata
    kubernetes-csi/livenessprobe
    kubernetes-csi/node-driver-registrar
    kubernetes-csi/csi-driver-nfs
    kubernetes-csi/csi-driver-iscsi
    kubernetes-csi/lib-volume-populator
    kubernetes-csi/volume-data-source-validator
    kubernetes-sigs/sig-storage-local-static-provisioner
    kubernetes-sigs/nfs-ganesha-server-and-external-provisioner
    kubernetes-sigs/nfs-subdir-external-provisioner
    kubernetes-sigs/container-object-storage-interface-controller
    kubernetes-sigs/container-object-storage-interface-provisioner-sidecar
)

# Repos which should eventually enable cloud image builds but currently
# don't.
readonly BROKEN_REPOS=(
    kubernetes-csi/csi-proxy
)

cat >"${OUTPUT}" <<EOF
# Automatically generated by k8s-staging-sig-storage-gen.sh.

postsubmits:
EOF

for repo in "${REPOS[@]}" "${BROKEN_REPOS[@]}"; do
    IFS=/ read -r org repo <<<"${repo}"
    default_branch="master"
    # external-snapshot-metadata repo uses "main" as the default branch.
    if [ "${repo}" == "external-snapshot-metadata" ]; then
    default_branch="main"
    fi
    cat >>"${OUTPUT}" <<EOF
  ${org}/${repo}:
    - name: post-${repo}-push-images
      rerun_auth_config:
        github_team_slugs:
          - org: kubernetes
            slug: release-managers
          - org: kubernetes
            slug: test-infra-admins
          - org: kubernetes
            slug: sig-storage-image-build-admins
      cluster: k8s-infra-prow-build-trusted
      annotations:
        testgrid-dashboards: sig-storage-image-build, sig-k8s-infra-gcb
      decorate: true
      decoration_config:
        timeout: 240m
        grace_period: 15m
      branches:
        # For publishing canary images.
        - ^${default_branch}$
        - ^release-
        # For publishing tagged images. Those will only get built once, i.e.
        # existing images are not getting overwritten. A new tag must be set to
        # trigger another image build. Images are only built for tags that follow
        # the semver format (regex from https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string).
        - ^v(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$
      spec:
        serviceAccountName: gcb-builder
        containers:
          - image: gcr.io/k8s-staging-test-infra/image-builder:v20241015-ff9ecc4d73
            command:
              - /run.sh
            args:
              # this is the project GCB will run in, which is the same as the GCR
              # images are pushed to.
              - --project=k8s-staging-sig-storage
              # This is the same as above, but with -gcb appended.
              - --scratch-bucket=gs://k8s-staging-sig-storage-gcb
              - --env-passthrough=PULL_BASE_REF
              - .
EOF
done

cat >>"${OUTPUT}" <<EOF

# Canary images are used by some Prow jobs to ensure that the upcoming releases
# of the sidecars work together. We don't promote those canary images.
# To avoid getting them evicted from the staging area, we have to rebuild
# them periodically. One additional benefit is that build errors show up
# in the sig-storage-image-build *before* tagging a release.
#
# Periodic jobs are currently only specified for the "master" branch
# which produces the "canary" images. While other branches
# could produce release-x.y-canary images, we don't use those.
periodics:
EOF

for repo in "${REPOS[@]}"; do
    IFS=/ read -r org repo <<<"${repo}"
    default_branch="master"
    # external-snapshot-metadata repo uses "main" as the default branch.
    if [ "${repo}" == "external-snapshot-metadata" ]; then
    default_branch="main"
    fi
cat >>"${OUTPUT}" <<EOF
  - name: canary-${repo}-push-images
    cluster: k8s-infra-prow-build-trusted
    annotations:
      testgrid-dashboards: sig-storage-image-build, sig-k8s-infra-gcb
    decorate: true
    interval: 168h # one week
    extra_refs:
      # This also becomes the current directory for run.sh and thus
      # the cloud image build.
      - org: ${org}
        repo: ${repo}
        base_ref: ${default_branch}
    spec:
      serviceAccountName: gcb-builder
      containers:
        - image: gcr.io/k8s-staging-test-infra/image-builder:v20241015-ff9ecc4d73
          command:
            - /run.sh
          env:
            # We need to emulate a pull job for the cloud build to work the same
            # way as it usually does.
            - name: PULL_BASE_REF
              value: ${default_branch}
          args:
            # this is the project GCB will run in, which is the same as the GCR
            # images are pushed to.
            - --project=k8s-staging-sig-storage
            # This is the same as above, but with -gcb appended.
            - --scratch-bucket=gs://k8s-staging-sig-storage-gcb
            - --env-passthrough=PULL_BASE_REF
            - .
EOF
done

[hairyhum]

This will break jobs for all other sidecars.

@xing-yang can you elaborate? I didn't remove master from there, just added main and there are already other templates for branches which may not exist.

[hairyhum]

@Rakshith-R added the check to the canary image job generation.

[Rakshith-R]

Same logic that is implemented below is required here as well.

It looks good to me after this last change 🎉 !

Thanks

[Rakshith-R]

thanks,
Looks good to me.

[Rakshith-R]

/cc @xing-yang

[hairyhum]

@xing-yang can you please review?

[xing-yang]

/lgtm
/approve

[xing-yang]

/assign @dims @BenTheElder @ameukam

[dims]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dims, hairyhum, xing-yang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• config/jobs/image-pushing/OWNERS [dims]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

@hairyhum: Updated the job-config configmap in namespace default at cluster test-infra-trusted using the following files:

• key k8s-staging-sig-storage.yaml using file config/jobs/image-pushing/k8s-staging-sig-storage.yaml

In response to this:

Follow-up to #33303

Addressing #33303 (comment)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.