Build and release Geoserver multi-platform for ARM64 and AMD64.

[alfonx]

Currently, the docker images at https://docker.osgeo.org are only compatible with AMD64 platform. Basically, they run on Intel and AMD, not on ARM.

ARM CPUs are winning in popularity, and hosters like Hetzner offer them as root-servers, too.

This pull request will change the release.sh to build and push images for both platforms in parallel. After applying this PR, the release-Script will fail unless the docker-daemon has been set up for multi-platform build, as described here
https://docs.docker.com/build/building/multi-platform/#qemu

tl/tr:

• Add to /etc/docker/daemon.json {"features": {"containerd-snapshotter": true }} and restart docker, e.g. systemctl restart docker
• Create a custom builder:
  docker buildx create --name container-builder --driver docker-container --use --bootstrap

After that, running build/release.sh should build and push images for both platforms. I cannot test the push, since I have not Nexus Repository to test! Building does not take much longer.

Whoever is releasing new geoserver docker images regularly, will need a multi-platform docker setup.

NB: With our docker-repository provider quay.io, we experienced, that the first push to a repository had to be multi-platform, for the repository to support multi-platform! I think we simply have to try with Nexus here.

Of course I am happy to test on ARM as soon and images are pushed.

[buehner]

Thank you for the PR. I really like the idea to extend the support to other platforms. I am only partially familiar with the build environment (Jenkins on build.geoserver.org), so I think we need support from others (with more rights to do such high level configuration). @jodygarnett could you help here regarding the QEMU steps described above?

[jodygarnett]

@tbarsballe do you have access to /etc/docker/daemon.json or do we contact GeoSolutions system admin...

[tbarsballe]

I do have access, but the command

docker buildx create --name container-builder --driver docker-container --use --bootstrap

Is failing. Not sure if it's a version issue (we're on Docker 24.0.5), or some missing prerequisite.

[tbarsballe]

Though, reading through the linked documentation, it sounds like we need to to enable the containerd image store (which I have done) OR create a custom builder, not both, so we might be good to go.

[alfonx]

Thanks @tbarsballe .. I think further steps could look like this:

• Change the Jenkins-Job to do "./release.sh build" instead of "./release.sh buildandpublish "
• Merge the PR
• Check that it builds in the jenkins job
• Get hold of someone with r/w access to https://docker.osgeo.org/ so that we could delete/correct any broken pushs
• Try to run "./release.sh buildandpublish" any test pulling and starting on ARM64 + AMD64

[jodygarnett]

I am checking in with this activity for 2.26.0 release. I am only releasing locally as something is troubling Jenkins (see geoserver-devel email list).

Could I merge this and do a manual release with ARM64?

[alfonx]

Could I merge this and do a manual release with ARM64?

Yes.

[buehner]

It seems that the renovate bots of geoserver users are suggesting to update to geoserver 2.26.0 as the docker image is already published: https://docker.osgeo.org/#browse/browse:geoserver-docker:v2%2Fgeoserver%2Ftags%2F2.26.0

I tried docker pull docker.osgeo.org/geoserver:2.26.0 (on my AMD64 machine) and then i had a look at docker image inspect docker.osgeo.org/geoserver:2.26.0 and realized: "Architecture": "arm64" (where 2.25.3 still has amd64)

I guess this is somehow related to this MR? @jodygarnett @tbarsballe

I am just asking myself what exactly happened? Is it a different base configuration on jenkins? Or the result of a manual action?

How will we fix this?

[jodygarnett]

Enabling this option to try out locally:

However it fails with:

failed commit on ref "layer-sha256:f05cc2820fbb694743afaba6d546d882af7fa0f1143f374f8c2e1fe104da7af0": unexpected status from PUT request to https://geoserver-docker.osgeo.org/v2/geoserver/blobs/uploads/fa4af31e-30b7-4d16-a781-1ae0d9d2dfe0?digest=sha256%3Af05cc2820fbb694743afaba6d546d882af7fa0f1143f374f8c2e1fe104da7af0: 413 Request Entity Too Large

We will have to ask the system admin committee to adjust web server limit for upload.

[tbarsballe]

Independant from the above issue, it appears that enabling the containerd store on Jenkins broke the existing builds (for example) as follows:

parent snapshot sha256:334047a74abd6cc7c7191ab195b9a909f4ff24f50ee6f82cb9780fc8ca8a82a5 does not exist: not found

(The actual snapshot sha changes each build, so it appears to be an intermediate snapshot)

While this error makes some amount of sense as containerd images are incompatible with standard images, removing all stored images from the server was not able to resolve this failure, so something else seems to be going on.

Reverting the change to daemon.json and disabling the containerd store fixed the build failure, so without a resolution to the above issue I don't think we can move forward with this change.