CRI-O: match version against openshift

[runcom]

@mrunalp @Kargakis @stevekuznetsov PTAL this will fix openshift/origin#17546 by pulling the correct image for the CRI-O system container

Signed-off-by: Antonio Murdaca runcom@redhat.com

[runcom]

Tags live here for now: https://hub.docker.com/r/runcom/cri-o-system-container/tags/ - we'll configure docker.io/crio/cri-o to rebuild those tags when needed and make the required changes here as well (cri-o/cri-o#1201)

[stevekuznetsov]

Historically logic like this has a habit of breaking whenever something crazy happens with versioning. I think it may be better to have a switch-case and very explicit requirements?

[runcom]

You mean to not concatenate strings to build up the image string for the installer? We can avoid that, but that requires us to update this repo every time a new release come up no? I think I'm missing something

[stevekuznetsov]

Yes, it would require it to be updated. If you think releases are very frequent then maybe it's a bad idea. If it's once per release of Origin that's usually once every couple of months and totally OK. I would rather heave a static list/mapping we have to update than logic that fails in some weird edge cases or ends up installing something unexpected. Switch/case can have a well defined default

[ashcrow]

Is crio_tag being set? In output that @mrunalp passed over to @sdodson and I the end result looked like:

docker.io/runcom/cri-o-system-container:v

[giuseppe]

true. If this is the version currently used by the CI then crio_tag looks empty

[ashcrow]

This seems to be merged .... but not "merged". 😕

[mrunalp]

I just talked with @runcom He deployed it to test to see if it works. We need to debug why the tag isn't getting set.

[runcom]

yup, I'll debug it asap

[ashcrow]

@runcom were you able to find the issue?

[runcom]

I'm debugging here just right now, I'll update this once I find something

[runcom]

should be fixed now, I'll work on Steve's switch case for images

[runcom]

I've deployed this to the origin CI already so we can test if it fixes the empty crio_tag issue

[runcom]

@stevekuznetsov added the case statement for the crio image PTAL (deployed to the CI as well for the crio jobs)

[stevekuznetsov]

I was also interested in this being fragile. Can you use regex in the case statements?

[runcom]

You mean extracting something like "3.9" with a regexp from that file? I could do that yeah

[stevekuznetsov]

Right

[runcom]

fixed, now using grep

[runcom]

@ashcrow so I've fixed the image tag but now we have another issue https://ci.openshift.redhat.com/jenkins/job/test_branch_origin_extended_conformance_crio/185/consoleFull#54402166758b6e51eb7608a5981914356

which is not related to CRI-O

Failure summary:


  1. Hosts:    localhost
     Play:     OpenShift Health Checks
     Task:     Run health checks (install) - EL
     Message:  One or more checks failed
     Details:  check "docker_image_availability":
               One or more required container images are not available:
                   openshift/origin-docker-registry:a5c8037
               Checked with: skopeo inspect [--tls-verify=false] [--creds=<user>:<pass>] docker://<registry>/<image>
               Default registries searched: docker.io

[stevekuznetsov]

Are you starting the job manually? Use /test. The job expects many parameters to be set that were not provided in that run.

[runcom]

Are you starting the job manually? Use /test. The job expects many parameters to be set that were not provided in that run.

oh, I see, I'll just /test crio then

[runcom]

So it's failing again https://ci.openshift.redhat.com/jenkins/job/test_pull_request_origin_extended_conformance_crio/427/consoleFull#-190934373858b6e51eb7608a5981914356

but this time it tries to restart CRI-O but it hasn't been installed, has something changed in openshift-ansible that now forgets to "atomic install" CRI-O?

@giuseppe @ashcrow

[ashcrow]

@runcom not that I'm aware of ... but looking at the code it doesn't look correct. It literally installed node items but didn't install the system container 😕.

I see that the docker role has been renamed container_runtime (which makes sense) ... I wonder if something slipped through the cracks.

[ashcrow]

I think this may be it but I'm not sure. The role used to require the docker role, but it no longer has a dependency.

@michaelgugino / @sdodson ^^ Thoughts?

[ashcrow]

Looks like it merged yesterday openshift/openshift-ansible#6297

[runcom]

So that's another reason to test cri-o as well in openshift ansible ha

[michaelgugino]

Docker has been renamed to container_runtime to more accurately reflect its purpose.

This must now be called from prerequisites.yml before calling byo/config.yml

[ashcrow]

@michaelgugino thanks. Was there a reason this wasn't called already or is it a bug from refactoring?

[giuseppe]

I have some patches here, I will push them shortly.

Another thing, we should enable container-engine in the system-containers tests.

[sdodson]

@michaelgugino thanks. Was there a reason this wasn't called already or is it a bug from refactoring?

It's intentional. So we need to ensure those vars are set when calling prerequisites too before this PR will produce the desired outcome.

[runcom]

It's intentional. So we need to ensure those vars are set when calling prerequisites too before this PR will produce the desired outcome.

isn't it all fixed now? we have a bunch of PR in both origin and openshift-ansible that are blocked on this cause they're using the wrong cri-o system container image (maybe I'm missing something)

[michaelgugino]

Any and all variables used with any and all of the plays must be used with prerequisites.yml.

In the case of container_runtime, that is all configured during prerequisites.yml. prerequisites.yml configures all of the host-level settings for each container runtime. In the case of system containers, this is when the runtime images are pulled and configured.

I highly recommend refactoring some of the tests to use a extra_vars file instead of passing most things via cli in the script. This will help alleviate some of the issues in the future.

[mrunalp]

@michaelgugino Is that something which can be done in a follow-on PR?

[ashcrow]

@sdodson++

[mrunalp]

@sdodson Thanks!

[mrunalp]

@michaelgugino Does it look okay now with @sdodson commit?

[sdodson]

Yeah, the job ran with latest on master so I think this change is good to go.

[ashcrow]

I opened up runcom/cri-o-system-container#1 to ensure that this is the image that is in @runcom's namespace. Pulling and looking at it I'm pretty sure it is.