Consolidate container images on Docker Hub and Quay.io

[vashirov]

Issue Description
We have images based on
openSUSE: https://hub.docker.com/r/389ds/dirsrv
Fedora/CentOS Stream: https://quay.io/repository/389ds/dirsrv

Since they have the same namespace/image:tag, users get different images depending on their tool of choice (docker vs. podman).

I think we should provide same images in both registries with proper tags, i.e.

389ds/dirsrv:fedora
389ds/dirsrv:c9s
389ds/dirsrv/opensuse
etc.

Not sure what should point to latest though.

@Firstyear, thoughts?

[johanneskastl]

I would use a similar approach other images are using, i.e. have the tags not only be the distribution, but a combination of version and distribution.

• 389ds/dirsrv:latest-fedora
• 389ds/dirsrv:latest-opensuse
• 389ds/dirsrv:latest-c9s
• 389ds/dirsrv:2.2.6-fedora
• 389ds/dirsrv:2.2.6-opensuse
• 389ds/dirsrv:2.2.6-c9s
• 389ds/dirsrv:2.2-fedora
• 389ds/dirsrv:2.2-opensuse
• 389ds/dirsrv:2.2-c9s

[Firstyear]

@vashirov I'm more than happy to standardise on one-or-the-other - Have I previously provided you admin access on the docker hub repos?

We should probably have a better idea of how we want to manage docker going forward as it's important for both rh and suse :)

[vashirov]

Hey @Firstyear! I don't have access to the Docker Hub repos. It would be great, if you could add me there :) My username is the same.

I also want to look again into using UBI or SLE BCI as the base images. Last time I checked, there were some missing dependencies, that's why there are c9s and fedora flavors instead.

I will make a new PoC repo (to avoid any disruption to the current repo users) with the new tagging/versioning scheme and share it here.

[Firstyear]

Okay sent you the docker invite. :)

SLE BCI is probably a good idea, we have BCI images for 389-ds (I think?) but I'm not 100% sure if they are public or not? I think the challenge with the SLE BCI images is that to "install" packages you need an active SUSE subscription (which I have through being an employee, but we don't have for dockerhub/community). So that would mean we'd need to build them in our internal build service (generally IBS compared to the opensuse build server aka OBS). So we could do them as BCI but we'd really just be "mirroring" these to hub.docker.com. Also not sure if that's allowed per content / license so I'd need to ask some questions internally.

So I think a PoC repo is a good idea, as well as knowing what we want the "tags" to look like and what we'd want to push. For example do we just want a single set of images on hub.docker.com just with versions and patches? Or do we want to actually split fedora / rh-centos / sle images with their own versions too?

[vashirov]

Thanks for the invite, accepted.

SLE BCI is freely available: https://registry.suse.com/bci/bci-base-15sp4/index.html and can be distributed: https://www.suse.com/products/base-container-images/FAQ/#can-i-freely-distribute-applications-built-on-bci?

I was able to pull it and install 389-ds from the repos there without any subscription. But it has an older version of 389-ds: 389-ds-2.0.17~git20.ff6dbd9-150400.3.23.1.x86_64.
If I understand the FAQ correctly, we can build newer versions of 389-ds on top of 15.4 in OBS, install it in SLE BCI and redistribute the resulting image.
But I see that there are dependency issues for 15.4 build: https://build.opensuse.org/package/show/network:ldap/389-ds

unresolvable: nothing provides libldap-data = 2.4.46-150200.14.11.2 needed by libldap-2_4-2, (got version 2.6.4-lp154.408.2)

[Firstyear]

We already have a BCI image you can re-publish though https://registry.suse.com/suse/389-ds/index.html

I need to check what version of 389-ds / SLE that container is based on though to be sure. I can speak with the BCI maintainers to see if we can get the labeled/tagged separately for 2.0 vs other versions.

The issue you are seeing with the libldap-data is because 15.4 is built with openldap-2.4, but network:ldap has 2.6. This means that libldap-data which isn't versioned is 2.6, but if anything in 15.4 links to libldap-2_4 it can't then access libldap-data that goes with it. To fix this we'd need to rename libldap-data in SLE but that's unlikely to happen due to the conservative package naming practices etc.

So I think if we wanted to build specific versions for containers to go into BCI we either need to do that as SUSE and then we can mirror those to hub.docker.com since we can build that in our internal build service. Or we'll need to setup a dedicated repo in OBS to do the versioned builds.

[Firstyear]

@vashirov Checking with some people inside SUSE we can't mirror BCI images on hub.docker.com.

[vashirov]

We already have a BCI image you can re-publish though https://registry.suse.com/suse/389-ds/index.html

So this is the downstream image, it contains (old) version of 389DS that comes from SLE repos.
As an upstream we should provide latest versions of 389DS.

I need to check what version of 389-ds / SLE that container is based on though to be sure. I can speak with the BCI maintainers to see if we can get the labeled/tagged separately for 2.0 vs other versions.

It's 389-ds-2.0.17~git20.ff6dbd9-150400.3.23.1.

The issue you are seeing with the libldap-data is because 15.4 is built with openldap-2.4, but network:ldap has 2.6. This means that libldap-data which isn't versioned is 2.6, but if anything in 15.4 links to libldap-2_4 it can't then access libldap-data that goes with it. To fix this we'd need to rename libldap-data in SLE but that's unlikely to happen due to the conservative package naming practices etc.

Yeah, these kind of dependency issues I alluded to before. UBI also has a missing dependency for which I filed a bugzilla. If it won't be resolved, I'm leaning to use c9s as a single base image since it doesn't have dependency issues and we can have different versions of 389DS build for it in COPR.

So I think if we wanted to build specific versions for containers to go into BCI we either need to do that as SUSE and then we can mirror those to hub.docker.com since we can build that in our internal build service. Or we'll need to setup a dedicated repo in OBS to do the versioned builds.

I think there is some misunderstanding. I don't want to redistribute BCI images as they are, with the content from BCI repos. But rather use BCI/UBI as our base image, build 389DS in OBS/COPR to target the relevant chroots.

@vashirov Checking with some people inside SUSE we can't mirror BCI images on hub.docker.com.

https://www.suse.com/products/base-container-images/FAQ/#does-bci-let-me-distribute-my-container-images-anywhere-i-want?

Does BCI let me distribute my container images anywhere I want?
Yes, SUSE will never oversee what you do with your images and how you distribute them. BCIs are freely distributable, and you can also distribute your applications as you want if you comply with the EULA.

EULA: https://www.suse.com/licensing/eula/download/sbci/suse-base-container-image-licence-en.pdf

Relevant part:

DISTRIBUTION LICENSE
Subject to compliance with the terms and conditions of this Agreement, Licensor grants to
You a perpetual, irrevocable, non-exclusive, worldwide license to reproduce, use and
distribute SUSE Linux Enterprise Base Container Images (including installable packages
from the SUSE Linux Enterprise Base Container Image Repository) of the Software listed on
the SUSE Registry marked as SLE BCI where (a) such images are used as a first and separate
layer of a derived container image and (b) the SUSE Linux Enterprise Base Container Images
and any added packages from the SUSE Linux Enterprise Base Container Image Repository
(as preconfigured in the SUSE Linux Enterprise Base Container Images) are unmodified and
(c) You include this license in your own derived product which provides protection to the
Software equivalent to this Agreement, built on top of any SLE BCI container images and (d)
provided You do not make any statements on behalf of SUSE, including but not limited to,
stating or in any way suggesting (whether written or verbal) that SUSE supports or endorses
software built and delivered with a SUSE Linux Enterprise Base Container Image unless
agreed by SUSE.

IANAL, but it looks like I can redistribute a container image that uses BCI as the base, with our own repos/content on top of it. Am I wrong?

Anyways, this and the dependency issues make BCI and UBI images less viable options, unfortunately.

[Firstyear]

I'll have to double check about this with product management to be sure, but your reading of this text seems correct.

Yes, the dependency issue is annoying. I think that's just an artefact of network:ldap - if we wanted to do sle bci + multiple versions of 389-ds, then we'll need to take a different approach IMO to create these that avoids that OBS repo.

[Firstyear]

@vashirov For now, what structure of tags should we aim for? I think once we know that then we can work out the specifics of images later.

[vashirov]

I talked to some folks internally and I was pointed to sclorg as the example that we might want to follow.

Proposed structure:

(I'm using semver MAJOR.MINOR.PATCH notation here)

[registry]/dirsrv-${MAJOR_VERSION}${MINOR_VERSION}-${BASE_IMAGE}

That would expand to:

[registry]/dirsrv-22-c9s
[registry]/dirsrv-23-c9s
[registry]/dirsrv-24-c9s

[registry]/dirsrv-22-fedora
[registry]/dirsrv-23-fedora
[registry]/dirsrv-24-fedora

[registry]/dirsrv-22-leap
[registry]/dirsrv-23-leap
[registry]/dirsrv-24-leap

etc.

Let's take dirsrv-23-c9s as an example. It should have the following tags:

• Stable tags, i.e. when a new release is cut, and 389-ds-base git commit hash from which it was built.

dirsrv-23-c9s:v2.3.0 -> :3db81913e27002ced7df00e5625b804457593efb
dirsrv-23-c9s:v2.3.1 -> :6ad8f0752a515f758213e071b1a956f13ca920bb
dirsrv-23-c9s:v2.3.2 -> :e53751013284fd99f2e41e0ef44a2375a9b11bc4
dirsrv-23-c9s:v2.3.3 -> :c4b939b6d33dab855ef87d4a77e5693c60aa1874

• Floating tags for each minor version that follows git branch

dirsrv-23-c9s:latest -> :c4b939b6d33dab855ef87d4a77e5693c60aa1874
dirsrv-23-c9s:v2.3 -> :v2.3.3 -> :c4b939b6d33dab855ef87d4a77e5693c60aa1874

So if a user wants to use a specific patch version, they should use v2.3.2.
If they want to follow minor version, they should use v2.3, which will automatically upgrade to the next stable patch release.
And if they want to test latest minor version from git, they should use latest.

Now to the building part. We have COPR repos that follow Fedora releases and not 389DS versions. I heard complaints from users that they don't want to upgrade their instances to the next minor version, especially when we don't guarantee backward compatibility between minor versions. So we should have a similar version structure:

@389ds/389-directory-server -> latest stable from Fedora, i.e. 2.3 ATM.
@389ds/389-directory-server-2.2
@389ds/389-directory-server-2.3
@389ds/389-directory-server-2.4
@389ds/389-directory-server-next -> latest from Rawhide, i.e. 2.4 ATM.

And then we can use these repos to build our container images, including multiarch support.

[johanneskastl]

So if a user wants to use a specific patch version, they should use v2.3.2. If they want to follow minor version, they should use v2.3, which will automatically upgrade to the next stable patch release. And if they want to test latest minor version from git, they should use latest.

That proposal sounds really promising. This would allow helm charts to use the patch versions to allow rollbacks.

Thanks for taking care. I really appreciate this!

[Firstyear]

It's going to be a bit of a challenge for us with the git-commit hash when it comes to the bci images I think. We have an internal service that we use to build branches like 2.3 at any git patch level rather than specifically releasing at the minor point releases.

So we likely could do something like

dirsrv-23-sle:latest
dirsrv-23-sle:{minor}-{patches}
dirsrv-23-sle:1-16

I think a question is what should we do for the dirsrv:latest image, since that will be the common and first contact point for a lot of people. I'm happy for it to stay on tumbleweed and tracking the "absolute latest" of the project, but that may not be what you want.

[vashirov]

I was thinking of setting up additional GH actions to submit builds to COPR (which I just found out also supports leap and tumbleweed chroots), and then use GH actions in https://github.com/389ds/ds-container to build container images. Since this is an upstream project, I'd like to build everything in public.

As for dirsrv:latest, I think we should deprecate it and steer users to versioned repos.

[Firstyear]

The problem IMO is that :latest is a default in docker - people can search say:

 # docker search dirsrv
NAME                 DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
389ds/dirsrv         389 Directory Server Container                  13

And they won't be shown the tags, they'll go to "docker pull 389ds/dirsrv" which implies :latest. So I think we should continue to have :latest and just decide what it should be. Even more important here is there are likely already people using that tag, so we should continue to provide updates to it.

I think gh actions to build automatically is a good idea. It would save a lot of hassle. Would the gh actions be part of this repo or a seperate repos in 389ds org?

[vashirov]

:latest is default, yes. But what I meant is to deprecate 389ds/dirsrv altogether (including :latest) and instead redirect users to use 389ds/dirsrv-23-{c9s,fedora,sle,etc}, for example. It gives more control to choose the exact version, not worrying that :latest might be bleeding edge version from main git branch.
So if people search for dirsrv, they would find versioned repos without issues, as it would be a part of repo name.
The deprecation shouldn't be immediate, of course. Proper announcements, documentation updates should be done. Since 389ds/dirsrv:latest is 2.4 already, maybe we should leave it at that, and make 2.5 in versioned repo only. I'm open to suggestions.

I currently have GH Actions in a separate repo, it runs a nightly cron job. But to integrate it with repo events such as tagging and branching, it would be easier to have these actions in the main 389-ds-base repo.

BTW, thanks for adding tags to Docker Hub repo! I'm working on other things this week, but I hope to get back to this ASAP.

[Firstyear]

I think that just by convention and how docker works, it's a good idea to have a :latest. It's the default for people, and it will let people just keep upgrading over time. So I think we should actually really consider how we want to provide this.

I think the gh actions being here would be great, sounds good to me :)