Docker - upgrade to Debian Buster

[guidoiaquinti]

Description

Upgrade Docker images from Debian 9 (stretch) to Debian 10 (Buster). This PR was highly inspired by #6129 (happy to close this if Morgan wants to complete his PR).

This PR also includes the following changes:

• move from openjdk-8-jdk to default-jdk
• removed misc dependencies no longer need
• install maven and etcd from repos

Why

Debian Stretch is EOL since 2020-07-06 and Buster was released in July 2019. This deprecation is also blocking us from upgrading to go 1.15 #6792

Test plan

I'm not sure if it's enough but the build of the bootstrap images completes ✅

Details

➜  vitess git:(buster) ✗ make docker_bootstrap
building bootstrap image: common
[+] Building 132.7s (13/13) FINISHED
 => [internal] load .dockerignore                                                               0.0s
 => => transferring context: 35B                                                                0.0s
 => [internal] load build definition from Dockerfile.common                                     0.0s
 => => transferring dockerfile: 1.63kB                                                          0.0s
 => [internal] load metadata for docker.io/library/golang:1.13-buster                           1.3s
 => CACHED [1/8] FROM docker.io/library/golang:1.13-buster@sha256:66a3f6817c129f48c9cc9b96816a  0.0s
 => [internal] load build context                                                               0.0s
 => => transferring context: 2.52kB                                                             0.0s
 => [2/8] RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-instal  86.4s
 => [3/8] COPY bootstrap.sh dev.env build.env go.mod go.sum /vt/src/vitess.io/vitess/           0.1s
 => [4/8] COPY config /vt/src/vitess.io/vitess/config                                           0.0s
 => [5/8] COPY tools /vt/src/vitess.io/vitess/tools                                             0.0s
 => [6/8] RUN groupadd -r vitess && useradd -r -g vitess vitess &&     mkdir -p /vt/vtdataroot  0.4s
 => [7/8] RUN cd /vt/src/vitess.io/vitess &&  su vitess -c "/usr/local/go/bin/go mod download  23.1s
 => [8/8] RUN su vitess -c "mkdir -p /vt/src/vitess.io/vitess/bin && rm -rf /vt/bin && ln -s /  0.6s
 => exporting to image                                                                         20.5s
 => => exporting layers                                                                        20.5s
 => => writing image sha256:78d98e5dd8de19995cdedacd9be5792d359d5fb6ddbe5f00132837489dfc887d    0.0s
 => => naming to docker.io/vitess/bootstrap:common                                              0.0s
building bootstrap image: mariadb
[+] Building 138.6s (8/8) FINISHED
 => [internal] load build definition from Dockerfile.mariadb                                    0.0s
 => => transferring dockerfile: 1.00kB                                                          0.0s
 => [internal] load .dockerignore                                                               0.0s
 => => transferring context: 35B                                                                0.0s
 => [internal] load metadata for docker.io/vitess/bootstrap:common                              0.0s
 => [1/4] FROM docker.io/vitess/bootstrap:common                                                0.0s
 => => resolve docker.io/vitess/bootstrap:common                                                0.0s
 => [2/4] RUN for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --recv  23.2s
 => [3/4] WORKDIR /vt/src/vitess.io/vitess                                                      0.0s
 => [4/4] RUN ./bootstrap.sh                                                                  113.1s
 => exporting to image                                                                          2.1s
 => => exporting layers                                                                         2.0s
 => => writing image sha256:684cf2f68ed14d4b98db7705c48e8fee5cac172fbe96cc13edca92180eb29f86    0.0s
 => => naming to docker.io/vitess/bootstrap:mariadb                                             0.0s
building bootstrap image: mariadb103
[+] Building 165.2s (8/8) FINISHED
 => [internal] load build definition from Dockerfile.mariadb103                                 0.0s
 => => transferring dockerfile: 611B                                                            0.0s
 => [internal] load .dockerignore                                                               0.0s
 => => transferring context: 35B                                                                0.0s
 => [internal] load metadata for docker.io/vitess/bootstrap:common                              0.0s
 => CACHED [1/4] FROM docker.io/vitess/bootstrap:common                                         0.0s
 => [2/4] RUN apt-key adv --no-tty --recv-keys --keyserver keyserver.ubuntu.com 0xF1656F24C74  42.6s
 => [3/4] WORKDIR /vt/src/vitess.io/vitess                                                      0.0s
 => [4/4] RUN ./bootstrap.sh                                                                  120.4s
 => exporting to image                                                                          2.1s
 => => exporting layers                                                                         2.1s
 => => writing image sha256:c6cef2f5804658c1326d77fe2eb8c08a2cc42c91ce9e58e5f2a5f3227a411ce9    0.0s
 => => naming to docker.io/vitess/bootstrap:mariadb103                                          0.0s
building bootstrap image: mysql56
[+] Building 143.9s (8/8) FINISHED
 => [internal] load .dockerignore                                                               0.0s
 => => transferring context: 35B                                                                0.0s
 => [internal] load build definition from Dockerfile.mysql56                                    0.0s
 => => transferring dockerfile: 1.39kB                                                          0.0s
 => [internal] load metadata for docker.io/vitess/bootstrap:common                              0.0s
 => CACHED [1/4] FROM docker.io/vitess/bootstrap:common                                         0.0s
 => [2/4] RUN for i in $(seq 1 10); do apt-key adv --no-tty --recv-keys --keyserver pool.sks-  23.2s
 => [3/4] WORKDIR /vt/src/vitess.io/vitess                                                      0.0s
 => [4/4] RUN ./bootstrap.sh                                                                  118.3s
 => exporting to image                                                                          2.3s
 => => exporting layers                                                                         2.3s
 => => writing image sha256:8ff377036cd82787a2b8532f35be431c10605a272c160e445cb39e14aeab62f2    0.0s
 => => naming to docker.io/vitess/bootstrap:mysql56                                             0.0s
building bootstrap image: mysql57
[+] Building 216.1s (8/8) FINISHED
 => [internal] load build definition from Dockerfile.mysql57                                    0.0s
 => => transferring dockerfile: 1.27kB                                                          0.0s
 => [internal] load .dockerignore                                                               0.0s
 => => transferring context: 35B                                                                0.0s
 => [internal] load metadata for docker.io/vitess/bootstrap:common                              0.0s
 => CACHED [1/4] FROM docker.io/vitess/bootstrap:common                                         0.0s
 => [2/4] RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-instal  29.9s
 => [3/4] WORKDIR /vt/src/vitess.io/vitess                                                      0.0s
 => [4/4] RUN ./bootstrap.sh                                                                  183.5s
 => exporting to image                                                                          2.6s
 => => exporting layers                                                                         2.6s
 => => writing image sha256:6bac5de811e65e888ff032cdb0fc636e1f197d56d4698a3c5c0bcdc5e47a7c42    0.0s
 => => naming to docker.io/vitess/bootstrap:mysql57                                             0.0s
building bootstrap image: mysql80
[+] Building 180.7s (8/8) FINISHED
 => [internal] load build definition from Dockerfile.mysql80                                    0.0s
 => => transferring dockerfile: 1.19kB                                                          0.0s
 => [internal] load .dockerignore                                                               0.0s
 => => transferring context: 35B                                                                0.0s
 => [internal] load metadata for docker.io/vitess/bootstrap:common                              0.0s
 => CACHED [1/4] FROM docker.io/vitess/bootstrap:common                                         0.0s
 => [2/4] RUN for i in $(seq 1 10); do apt-key adv --no-tty --recv-keys --keyserver ha.pool.s  58.9s
 => [3/4] WORKDIR /vt/src/vitess.io/vitess                                                      0.0s
 => [4/4] RUN ./bootstrap.sh                                                                  118.2s
 => exporting to image                                                                          3.5s
 => => exporting layers                                                                         3.5s
 => => writing image sha256:39fb8500858388aa5080a92034239cffe18f5f7c31de49b6f10e6e9d89e9c8e2    0.0s
 => => naming to docker.io/vitess/bootstrap:mysql80                                             0.0s
building bootstrap image: percona
[+] Building 167.5s (8/8) FINISHED
 => [internal] load build definition from Dockerfile.percona                                    0.0s
 => => transferring dockerfile: 1.42kB                                                          0.0s
 => [internal] load .dockerignore                                                               0.0s
 => => transferring context: 35B                                                                0.0s
 => [internal] load metadata for docker.io/vitess/bootstrap:common                              0.0s
 => CACHED [1/4] FROM docker.io/vitess/bootstrap:common                                         0.0s
 => [2/4] RUN for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --recv  35.0s
 => [3/4] WORKDIR /vt/src/vitess.io/vitess                                                      0.0s
 => [4/4] RUN ./bootstrap.sh                                                                  130.6s
 => exporting to image                                                                          1.8s
 => => exporting layers                                                                         1.8s
 => => writing image sha256:0cb3fd21184053b63937a93aa964f7edef12d5356c97d9bba805f3bc6f4e97fb    0.0s
 => => naming to docker.io/vitess/bootstrap:percona                                             0.0s
building bootstrap image: percona57
[+] Building 141.3s (8/8) FINISHED
 => [internal] load build definition from Dockerfile.percona57                                  0.0s
 => => transferring dockerfile: 894B                                                            0.0s
 => [internal] load .dockerignore                                                               0.0s
 => => transferring context: 35B                                                                0.0s
 => [internal] load metadata for docker.io/vitess/bootstrap:common                              0.0s
 => CACHED [1/4] FROM docker.io/vitess/bootstrap:common                                         0.0s
 => [2/4] RUN for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --recv  33.6s
 => [3/4] WORKDIR /vt/src/vitess.io/vitess                                                      0.0s
 => [4/4] RUN ./bootstrap.sh                                                                  105.3s
 => exporting to image                                                                          2.4s
 => => exporting layers                                                                         2.3s
 => => writing image sha256:082a38ec46cd1a114ceffc4b7b44cbd170efb68e288227cb2f99fbf2eaac4801    0.0s
 => => naming to docker.io/vitess/bootstrap:percona57                                           0.0s
building bootstrap image: percona80
[+] Building 613.9s (8/8) FINISHED
 => [internal] load build definition from Dockerfile.percona80                                  0.0s
 => => transferring dockerfile: 1.58kB                                                          0.0s
 => [internal] load .dockerignore                                                               0.0s
 => => transferring context: 35B                                                                0.0s
 => [internal] load metadata for docker.io/vitess/bootstrap:common                              0.0s
 => CACHED [1/4] FROM docker.io/vitess/bootstrap:common                                         0.0s
 => [2/4] RUN for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --rec  504.1s
 => [3/4] WORKDIR /vt/src/vitess.io/vitess                                                      0.0s
 => [4/4] RUN ./bootstrap.sh                                                                  105.0s
 => exporting to image                                                                          4.7s
 => => exporting layers                                                                         4.7s
 => => writing image sha256:8d657e9ac443da406417326e70b10f7d0be0e5f64032e40d8f1fbed1b9319bda    0.0s
 => => naming to docker.io/vitess/bootstrap:percona80                                           0.0s

[guidoiaquinti]

Not sure if we need this at all. I've only backported it from #6129

[deepthi]

@harshit-gangal should be ok to change the jdk version from openjdk-8 to default-jdk?

[harshit-gangal]

we need to keep it at jdk8 otherwise java code will not compile. Any reason for changing this?

[guidoiaquinti]

we need to keep it at jdk8 otherwise java code will not compile. Any reason for changing this?

Unfortunately there isn't openjdk-8-jdk available for Buster.

In stretch, default-jdk points to the Java runtime, or Java compatible development kit recommended for this architecture, which is openjdk-8-jdk for amd64 (see doc).

In buster, default-jdk points to the Java runtime, or Java compatible development kit recommended for this architecture, which is openjdk-11-jdk for amd64 (see doc).

[guidoiaquinti]

This is technically not needed as orchestrator has its own build process but I think it's 👍 if we align the versions across the whole repo.

[guidoiaquinti]

libperconaserverclient18.1-dev doesn't exist in Percona/buster

[derekperkins]

LGTM. I started publishing buster based images for the helm dockerfiles at 7.0. I've never worked with bootstrap, so I'm not the right person to approve any of those dependency changes.

[deepthi]

@dkhenry wdyt? Looks fine to me but I'd like to defer to you for final approval.

[dkhenry]

This is hitting the zookeeper error on build. Working through that now to finish testing

[dkhenry]

lgtm

[deepthi]

@guidoiaquinti can you cherry-pick this PR into release-8.0? we should do the same for #6792 once that is merged.