Containerfile: Move off of Fedora base image

[eskultety]

Background

We've historically based the cachi2 container on Fedora releases since it provides the latest toolchains we may need to be able to support latest versions of our list of supported package managers. A recent example would be go for which we'd need to wait for the next major Fedora release to actually get the latest 1.21. While that sounds plausible, we always need to keep an eye on Fedora release cycle so that we don't stay on an EOL'd release accidentally and because go's (or any other toolchain) release cycle isn't aligned with Fedora, we might need to wait up to 6 months (worst case scenario) to get the contents we need for upstream work.
Another thing to consider is that while Fedora provides the latest contents and maintainers are doing tremendous job at trying to keep everything smooth, it is also a not a distro known for stability, so in that regard providing a tool based on a stable and well maintained base image using strict rules for its curated contents may be a way forward.

Proposal

Given the connection to the App Studio project/org it might make sense for us to consider moving to one of the UBI 9 images from Red Hat (there are other options too), mainly because UBIs follow the same best practices as regular RHEL installations, but unlike those, UBIs are free to use for application containers.
While ^this definitely addresses the stability/security POV (which let's be fair hasn't been a problem for us so far), it does quite poorly with regards to getting fresh content - more specifically, we'd have to start pulling toolkits directly from their respective vendors content streams, e.g. curl https://go.dev/dl/go1.21.4.linux-amd64.tar.gz and install it directly.

Benefits

• stable base image
• strictly curated content
• well maintained

Drawbacks

• older versions of packages (which is the case with the 'old-latest' Fedora releases as well, but not nearly to the extent of UBIs or LTS releases in general)
• we may need to go and install some of our hard dependencies

Example solution

• here's a branch with changes necessary to move to UBI 9 (most notably downgrading the required nodejs version)
  example Dockerfile:

FROM registry.access.redhat.com/ubi9/ubi-minimal
LABEL maintainer="Red Hat"

WORKDIR /src
RUN microdnf -y install \
    --setopt install_weak_deps=0 \
    --nodocs \
    golang-bin \
    git-core \
    npm \
    python3 \
    python3-pip \
    && microdnf clean all

COPY . .

RUN pip3 install -r requirements.txt --no-deps --no-cache-dir --require-hashes && \
    pip3 install --no-cache-dir . && \
    # the git folder is only needed to determine the package version
    rm -rf .git

WORKDIR /src/js-deps
RUN npm install && \
    ln -s "${PWD}/node_modules/.bin/corepack" /usr/local/bin/corepack && \
    corepack enable yarn && \
    microdnf -y remove npm

ENTRYPOINT ["cachi2"]

• alternatively, just for fun I tried with Alpine 3.18 too just to see how much storage space we could save (turns out, about 200MB)
  example Dockerfile:

FROM docker.io/library/alpine:3.18
LABEL maintainer="Red Hat"

WORKDIR /src
RUN apk update &&\
    apk upgrade &&\
    apk add \
    go \
    git \
    npm \
    python3 \
    py3-pip

COPY . .

RUN pip3 install -r requirements.txt --no-deps --no-cache-dir --require-hashes && \
    pip3 install --no-cache-dir . && \
    # the git folder is only needed to determine the package version
    rm -rf .git

WORKDIR /src/js-deps
RUN npm install && \
    ln -s "${PWD}/node_modules/.bin/corepack" /usr/local/bin/corepack && \
    corepack enable yarn && \
    apk del npm

ENTRYPOINT ["cachi2"]

[ben-alkov]

it is also a not a distro known for stability

In my experience, this has not been the case since at least Fedora 30, at least as a workstation OS

[eskultety]

"stability" as in, features (APIs) are coming too fast sometimes (major versions often coming with breaking changes). It's good for development, not so much for a product IMO.

[eskultety]

ping. Can we resurrect this discussion on an alternative base image use since the original main premise of getting latest Go builds with Fedora ASAP has become (with recent Go developments) largely irrelevant? Technically, it should be possible to go with either Alpine or UBI9 with Alpine potentially offering more image size savings, however for the price of Go depending on musl libc implementation instead of glibc - this may be a complete non-issue for us, just mentioning it upfront. We could also consider a multi-stage build of the container image for further reduction of image size, e.g. Go depends on GCC but that's only due to go build which we'll likely won't need for the purposes of downloading deps, so we might be able to manage without; similarly with Yarn/NodeJS we could potentially achieve some size cuts there, keeping the image leaner.

[taylormadore]

I'm very much in favor of this proposal since it will help us get off of the fedora release train. As far as UBI vs. alpine, I'm not sure I have a strong opinion. I'd probably lean UBI just due to familiarity, but again, not a strong opinion.

[taylormadore]

This is a tangential topic, but it might also be nice to "pin" the digest of whichever container image we choose and have it updated in a deliberate manner by some tool (dependabot, renovate, etc.) if possible. I can remember a few times we've been burned in the past by surprise package updates in our images.

[eskultety]

Can you elaborate more on the proposed solution? I understand the motivation, but I'm not sure what you suggest can be done, especially for the base image, IOW whenever we build a new image, we'll pull the latest base image. I wonder how either renovate or dependabot could influence this in any scenario we care about.