cri-o 1.17.3 package doesnt exist anymore and v1.17.4 breaks minikube

[medyagh]

hello,

I would like to know how I could install cri-o v1.17.3 , since the package has vanished from the repo.

in minikube project, we faced a blocker to build the minikube base image since we can no longer install cri-o v1.17.3.
and unfortunately cri-o v1.17.4 doesn't not pass our tests. kubernetes/minikube#8861

we have invested a lot of testing infrastructure and engineering time to provide cri-o as a runtime in minikube, I would greatly apperciate it if you could assist us to have an easier way to install cri-o without a lot of efforts and suprises.

here is how we are installing it, if that helps to guide us better:

# install cri-o based on https://github.com/cri-o/cri-o/commit/96b0c34b31a9fc181e46d7d8e34fb8ee6c4dc4e1#diff-04c6e90faac2675aa89e2176d2eec7d8R128
RUN sh -c "echo 'deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_19.10/ /' > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list" && \    
    curl -LO https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/xUbuntu_19.10/Release.key && \
    apt-key add - < Release.key && apt-get update && \
    apt-get install -y --no-install-recommends cri-o-1.17

[haircommander]

hey @medyagh

Sorry for the frustration. I am not sure where 1.17.3 went, but I have attempted to package it (this is my first rodeo, so bear with me if there are any difficulties).

If you want to pin to a specific version, you can specify the subproject as so:
https://build.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:1.17:1.17.3

as oppposed to just 1.17 or pulling cri-o from the stable repo directly.

Right now, we're packaing for upstream as best we can, but we're a fairly small team who have a lot of requests flying at us. OBS is the best bet, as this is where we're putting our energy, unless you install from static binaries available at the releases page.

[medyagh]

@haircommander thanks for the response,
I tried this in our Dockerfile and it didn't build

RUN sh -c "echo 'deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.17:/1.17.3/xUbuntu_19.10/ /' > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list" && \
    curl -LO https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/xUbuntu_19.10/Release.key && \
    apt-key add - < Release.key && apt-get update && \
    apt-get install -y --no-install-recommends cri-o=1.17.3~3

here is the full log from the docker build

The following packages have unmet dependencies:
cri-o : Depends: containers-common (>= 0.1.27) but it is not installable
E: Unable to correct problems, you have held broken packages.

[haircommander]

you need to also add the top level repository:

echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /" > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | apt-key add -
apt install containers-common

[tstromberg]

Perhaps I am doing this wrong, but the v1.17.3~3 debian package seems to be missing nearly all of the files one would expect, and contains only a changelog.gz and copyright file:

# sudo dpkg -L cri-o
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/cri-o
/usr/share/doc/cri-o/changelog.gz
/usr/share/doc/cri-o/copyright

v1.17.3: https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.17:/1.17.3/xUbuntu_19.10/amd64/ shows a 5.5KB package

v1.17.4: https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.17:/1.17.4/xUbuntu_20.04/amd64/ shows a 20MB package.

[tstromberg]

The other quirk worth noting, in case it helps isolate the root cause, is that the v1.17.4 package is called 'cri-o-1.17', and the v1.17.3 package which is missing most of the files, is called 'cri-o'.

[haircommander]

I would sooner believe that I packaged it incorrectly than that 15mb difference is expected. I will try to look into it later today or tomorrow, kind of swamped rn

[tstromberg]

Thank you for the update (and your help)!

The inability to build a Docker image using crio v1.17.3 is blocking our minikube release, so we'll try in parallel to upgrade to v1.18.x to see if it addresses the strange disappearing pods after restart behavior we're seeing in v1.17.4

[haircommander]

as a side note, I'd like to hear more about this strange behaviour in 1.17.4, but I believe that's worthy of a new issue

[tstromberg]

as a side note, I'd like to hear more about this strange behaviour in 1.17.4, but I believe that's worthy of a new issue

Done: #4035

[afbjorklund]

@haircommander

I am not sure where 1.17.3 went

I think this is a limitation of OBS, we have the same problem with podman...

The only way to use this repository reliably is to cache the packages locally.

Previously I have been using bintray.io for this purpose, to set up a deb mirror.

https://dl.bintray.com/afbjorklund/podman/

https://dl.bintray.com/afbjorklund/cri-o/

Although I had to rebuild from source first, since the packages were already gone.

---

EDIT: Side note, bintray service allows hosting multiple versions without any issues.
But hosting the same version for multiple distributions doesn't work there, instead. :-(

The workaround is to add the distribution name to the version, to make them coexist.
There are different indexes for different distributions, so the problem is the file names.

[haircommander]

you can pin to a release by setting $VERSION to 1.17:1.17.3
that allows you to pin to a specific release, which will not update with the 1.17 project
https://build.opensuse.org/project/show/devel:kubic:libcontainers:stable:cri-o:1.17:1.17.3

[haircommander]

the subproject 1.17.3 is created, can y'all try it out and let me know if it solves your problems?

[afbjorklund]

It seems that crio was already upgraded to 1.18.3 instead: kubernetes/minikube@61583d6

[afbjorklund]

Prepared both 1.17.3 and 1.18.3 (requires new OS/go ISO), as an upgrade to current 1.17.1

• Upgrade crio to 1.17.3 kubernetes/minikube#8922

• Upgrade crio to 1.18.3 (merge after buildroot2020) kubernetes/minikube#8923 (squashed version of previous 1.18.1 revert)

[github-actions]

A friendly reminder that this issue had no activity for 30 days.