libvirt: only cache images in libvirt storage pool

[cgwalters]

I think the http cache code the installer carries is an artifact leftover from the V1 pipeline. Previously it was fetching an unversioned URL which could change.

Today, the installer does a 2-phase fetch and the URL of e.g. a -qemu.qcow2 contains a version number and further is by design immutable. The RHCOS pipeline will never overwrite a build.

So...the way I propose this work is that the installer drops the http cache code and instead downloads the image and then uploads it in to the libvirt pool (and ideally then deletes it from the cache). vagrant-libvirt does something like this.

I think the installer can assume it owns images starting with redhat-coreos?

[wking]

The current caching code is nicely generic, so you could swap in an alternative image source and still get caching. If we only want to support RHCOS and can commit to immutable filenames, then yeah, what you're proposing makes sense. And if signature checking for images is by signing meta.json (coreos/coreos-assembler#214), then we'll probably have to teach this code RHCOS-specifics anyway.

And @crawford has asked if we can get a latest-image URI back like we had before #554. Are we giving up on that? I'm fine without a latest-image URI, but want to make sure we're all on board ;).

[cgwalters]

If we only want to support RHCOS and can commit to immutable filenames,

I don't think there's a huge amount of logic in the "RHCOS pipeline" bits of the current code. We're talking about two HTTP requests to parse JSON. If we needed to do something else (and that's definitely a possibility; I can't say the current rhcos bits are final) we're not talking about a lot of code.

I think we should definitely commit though to immutable URLs at least when fetching via HTTP. But...people hacking locally with file:// may end up changing things.

And @crawford has asked if we can get a latest-image URI back like we had before #554.

The current code seems fine to me, right? A latest-image brings back all of the problems of caching that are avoided with the current scheme.

[wking]

I think we should definitely commit though to immutable URLs at least when fetching via HTTP.

The current caching implementation relies on immutable ETag -> content, and gives a single place where we could replace that with hash(content) -> content if we decide we don't trust ETags. Relying on immutable filenames instead doesn't seem like a big shift from an immutability perspective. The main changes are:

• Whether we cache into ~/.cache/openshift-install/libvirt/image for each new immutable version and push into libvirt volumes each time, vs. pushing straight into libvirt volumes.
• Whether we preserve previous header information in ~/.cache/openshift-install/libvirt/http. This is what allows us to efficiently cache headers regardless of whether the URI changes whenever the backing content changes.

But...people hacking locally with file:// may end up changing things.

We don't cache these people, we just push their image into libvirt for them and then delete the volume when we tear down the cluster. I don't think this would change, we'd just need Terraform, etc., to be able to handle both the "external volume" case for from-HTTP caches and "managed volume" case for from-file://.

[wking]

A latest-image brings back all of the problems of caching that are avoided with the current scheme.

What are the current problems of caching, besides issues with memory consumption in our current library (#645)?

[cgwalters]

What are the current problems of caching, besides issues with memory consumption in our current library

Lack of garbage collection. I already have to manage...a lot of different things that use images and containers.

And just general complexity.

[wking]

Lack of garbage collection.

Pushing the cached images into libvirt doesn't help with garbage collection, does it?

And just general complexity.

I don't see "supports HTTP cache control" as much of a mental complexity bump for users. I agree that the fact that we need to vendor (and help patch) an external library to support it is a maintainer complexity bump.

[cgwalters]

Pushing the cached images into libvirt doesn't help with garbage collection, does it?

I expect to look in libvirt's image pool for where my gigabytes have gone. ~/.cache yes but less so.

it is a maintainer complexity bump.

Yeah that's mostly what I meant.

(I wanted to refactor things to more clearly carry along the OS version from meta.json)