arm64 official docker images

[mcanevet]

Would it be possible to provide official arm64 images on DockerHub?

[csstaub]

Yeah, that should be doable in theory, though we use Docker Automated Builds for the images right now and afaict they don't support aarch64. So we'd have to build and release via Travis-CI or Github Actions maybe.

[mcanevet]

That would be great. This image is used by Prometheus Operator, and for now it is not usable on a multi-arch cluster (amd64, arm64 for my use case).

[csstaub]

If you happen to have examples of someone building aarch64 images in an automated workflow somewhere that we could take a look at that'd be great, that would help us set it up.

[mcanevet]

Here is what the guy from kube-webhook-certgen (also used by the prometheus-operator chart) did to fix the same issue I reported : jet/kube-webhook-certgen@758276e, I don't know which CI/CD tool he is using (workflow coded in the .cicd directory)

[csstaub]

More useful info I found: docker/hub-feedback#1261

[csstaub]

So, looking at kube-webhook-certgen, what they do is cross-compile static Go binaries and then use them to build distroless images. That'll be a bit trickier for us since we use CGo meaning we have a libc dependency for pkcs11 support. We could try to cross-compile an arm64 binary with CGo disabled, make it fully static, and then have that be the arm64 docker image (presumably docker+arm64+pkcs11 is not something anyone needs right now).

[csstaub]

So this should work:

❯❯❯ cat Dockerfile-arm64
# Dockerfile for squareup/ghostunnel, useful as a basis for other images.
#
# To build this image:
#     docker build -t squareup/ghostunnel .
#
# To run ghostunnel from the image (for example):
#     docker run --rm squareup/ghostunnel --version

FROM golang:1.13.3-alpine as build

MAINTAINER Cedric Staub "[email protected]"

# Dependencies
RUN apk add --no-cache --update gcc musl-dev libtool make git

# Copy source
COPY . /go/src/github.com/square/ghostunnel

# Build
RUN cd /go/src/github.com/square/ghostunnel && \
    GO111MODULE=on CGO_ENABLED=0 GOARCH=arm64 make clean ghostunnel && \
    cp ghostunnel /usr/bin/ghostunnel

# Create a multi-stage build with the binary
FROM arm64v8/alpine

COPY --from=build /usr/bin/ghostunnel /usr/bin/ghostunnel

ENTRYPOINT ["/usr/bin/ghostunnel"]

I don't have an arm64 machine to test this yet though, and I'm not sure how we'd get that built in Docker Hub alongside the regular image so we'd probably have to move building these to Travis-CI. Or maybe I could just release it manually until we automate it.

[gclawes]

Any movement on this? I've got an arm64 machine to test on if that helps.

[csstaub]

Unfortunately Docker Hub's automated builds doesn't support buildx at the moment, which makes it hard for us to do this in an automated fashion. The Dockerfile above should work to build your own image, though. But to provide it in Hub we'd either have to build and tag these manually on each release (probably not going to happen, sorry) or we'd have to invest the work into building our Docker images on a different build platform (maybe GitHub actions?).

[gclawes]

It looks like GitHub actions just supports self-hosed ARM runners: https://github.blog/changelog/2019-12-03-github-actions-self-hosted-runners-on-arm-architectures/

There does appear to be docker buildx support: https://github.com/marketplace/actions/docker-buildx

[rmb938]

You don't need buildx or your own runners to build multi-arch docker images. You can use https://github.com/multiarch/qemu-user-static to enable arm builds natively on the public runners.

However with that docker file posted above, no arm commands are being run so qemu magic isn't needed. It'll just work.

[mrueg]

Since travis is already in use, I guess one path forward would be to have travis push the images to dockerhub per release/commit.

Would that be an option?

[cablespaghetti]

Making some good progress with the Travis route. Should have an initial PR in a day or two. :)
https://travis-ci.com/github/cablespaghetti/ghostunnel/jobs/371442498#L932
https://hub.docker.com/r/cablespaghetti/ghostunnel/tags