use image_family instead of static image reference

[MHBauer]

The intention here seems to be to test on the stable version of cos.

The history of this file shows many other environments, such as coroes, containervm, different versions of ubuntu with different versions of docker.

Given that it hasn't been running on cos for 6 weeks or so, I'm not sure there's much signal in running on multiple cos versions. cos-stable image_family should float as we like. Alternatively, we can go on an lts image family, such as cos-77-lts.

This would effect these jobs as they reference this file:

config/jobs/cadvisor/cadvisor.yaml
42:        - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml --test-suite=cadvisor
98:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml --test-suite=cadvisor

config/jobs/kubernetes/sig-node/node-docker.yaml
18:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
47:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
75:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml

config/jobs/kubernetes/sig-node/sig-node-presubmit.yaml
36:        - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
145:        - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml

config/jobs/kubernetes/sig-node/node-kubelet.yaml
27:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
54:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
153:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
180:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
210:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml

config/jobs/kubernetes/node-problem-detector/node-problem-detector-ci.yaml
93:        "--node-args=--image-config-file=/home/prow/go/src/k8s.io/test-infra/jobs/e2e_node/image-config.yaml --extra-envs=${EXTRA_ENVS}"

config/jobs/kubernetes/node-problem-detector/node-problem-detector-presubmits.yaml
90:          "--node-args=--image-config-file=/home/prow/go/src/k8s.io/test-infra/jobs/e2e_node/image-config.yaml --extra-envs=${EXTRA_ENVS}"

config/jobs/kubernetes/sig-release/release-branch-jobs/1.18.yaml
123:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
155:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
1052:        - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml

config/jobs/kubernetes/sig-release/release-branch-jobs/1.15.yaml
86:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
118:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
757:        - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml

config/jobs/kubernetes/sig-release/release-branch-jobs/1.16.yaml
86:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
118:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
804:        - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml

config/jobs/kubernetes/sig-release/release-branch-jobs/1.17.yaml
123:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
155:      - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml
860:        - --node-args=--image-config-file=/workspace/test-infra/jobs/e2e_node/image-config.yaml

[MHBauer]

PTAL
/cc @vpickard @bart0sh @bsdnet

[k8s-ci-robot]

@MHBauer: GitHub didn't allow me to request PR reviews from the following users: bsdnet.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

PTAL
/cc @vpickard @bart0sh @bsdnet

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bsdnet]

cos-stable is a moving target, and it maybe change , fo example: it maybe cos-83 or cos-84.
If the purpose of the image is for pre-commit , I would suggest using LTS image/family instead.
To do step by step, maybe we rename cos-stable-77-12371-227-0 to cos-77-12371-227-0
first for now. It is a safe bet per my today's knowledge.

[bsdnet]

If we want to use family, cos-81-lts should be more proper.

[bart0sh]

@bsdnet

cos-stable is a moving target, and it maybe change , fo example: it maybe cos-83 or cos-84

Using cos-81-lts family can potentially break jobs. When gcp switches to cos-83-lts they may not provide cos-81-lts anymore. This is what will happen with cos-73-lts soon as it approaches EOL.

[bart0sh]

To do step by step, maybe we rename cos-stable-77-12371-227-0 to cos-77-12371-227-0
first for now. It is a safe bet per my today's knowledge.

We don't know that as we haven't tested it. Recently we did similar switch from cos-stable-73 to cos-73 and it broke one of the PR jobs.

[MHBauer]

should 1 be cos-stable and 2 be cos-XX-lts? ought to require rolling them less frequently?

[bsdnet]

@bsdnet

cos-stable is a moving target, and it maybe change , fo example: it maybe cos-83 or cos-84

Using cos-81-lts family can potentially break jobs. When gcp switches to cos-83-lts they may not provide cos-81-lts anymore. This is what will happen with cos-73-lts soon as it approaches EOL.
No, this won't. LTS is a long term supported. cos-83 will not become LTS.
You could see that from the past: cos-69, cos-73, cos-77, next LTS is cos-81.

[bsdnet]

should 1 be cos-stable and 2 be cos-XX-lts? ought to require rolling them less frequently?

Yes, if it is cos-xx-lts, and it is stable and less frequently. For cos-stable, it supposes every 6 weeks.
See https://cloud.google.com/container-optimized-os/docs/resources/support-policy

[vpickard]

should 1 be cos-stable and 2 be cos-XX-lts? ought to require rolling them less frequently?

Yes, if it is cos-xx-lts, and it is stable and less frequently. For cos-stable, it supposes every 6 weeks.
See https://cloud.google.com/container-optimized-os/docs/resources/support-policy

This seems to be a very reasonable approach for the cos images. I'd suggest updating this PR to do that. @bsdnet and Ning Liao (nick?) are working on the cos image update policy, we can adjust this again later if needed.

[vpickard]

From the conversation, I agree it would be good to have 1 be cos-stable and 2 be cos-XX-lts. Would be good to update the PR to reflect that.

Also, @bsdnet and Ning are creating/reviewing the cos image policy. We could update later if needed based on policy.

[bart0sh]

Have I understood correctly that we're discussing these 2 possible image maintenance policies?

• use image_family. This gives us less maintenance effort for the price of possible failures when COS changes image versions.
• continue using image. This gives us much less chances for tests to fail for the price of more maintenance effort. In this scenario test job maintainers should test new images and change image names in a job yaml manually when COS changes image versions.

[vpickard]

Have I understood correctly that we're discussing these 2 possible image maintenance policies?

• use image_family. This gives us less maintenance effort for the price of possible failures when COS changes image versions.
• continue using image. This gives us much less chances for tests to fail for the price of more maintenance effort. In this scenario test job maintainers should test new images and change image names in a job yaml manually when COS changes image versions.

Yes, that is the current thinking. I believe we should proceed with this approach. @bsdnet and Ning can share their thoughts on COS image policy, and we can adjust if needed going forward.

@bart0sh Does that sound reasonable to you?

[bart0sh]

@vpickard @bsdnet @MHBauer So, what would be our decision? Should we continue using 'image:', start using 'image-family:' or anything else? As far as I understood this proposal we should keep using 'image:' and implement automation around it.

[SergeyKanzhelev]

@bsdnet any updates on it?

With the current structure of tests - jobs/e2e_node/image-config.yaml is used in all type of tests, presubmits, release blocking and feature tests. Perhaps we should have separate image configuration files than? What do you think?

[bsdnet]

The doc is updated in https://github.com/kubernetes/kubernetes/tree/master/cluster/gce/gci

https://github.com/kubernetes/kubernetes/tree/master/cluster/gce/gci#what-image-is-needed-for-your-test specifies how to choose an image.

[karan]

+1 to running tests with image_family (except let's using cos-xx-lts instead of cos-stable). We should be certifying k8s for major releases of images (COS LTS in this case) and not specific images. I would also prefer to use that in presubmits and everywhere else to get as much test coverage as possible, and detect issues sooner (in presubmits).

[vpickard]

Based on the sig-node ci meeting today, the general consensus is to have one image for all of the job types (PR, CI, etc), and the preferred image would be to use the image_family (cos-81-lts, for example).

+1 to @karan comment above.

[SergeyKanzhelev]

/lgtm

[MHBauer]

/assign @dims @dashpole

[dims]

/approve
/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dims, MHBauer, SergeyKanzhelev

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• jobs/e2e_node/OWNERS [dims]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment