Skip to content

Docker - upgrade to Debian Buster #6833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
deepthi merged 1 commit into from
Oct 15, 2020
Merged

Docker - upgrade to Debian Buster #6833

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 5 additions & 18 deletions docker/bootstrap/Dockerfile.common
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,35 +1,22 @@
FROM golang:1.13-stretch
FROM golang:1.13-buster

# Install Vitess build dependencies
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
# TODO(mberlin): Group these to make it easier to understand which library actually requires them.
automake \
bison \
bzip2 \
ant \
chromium \
curl \
default-jdk \
etcd \
Copy link
Copy Markdown
Member Author

@guidoiaquinti guidoiaquinti Oct 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we need this at all. I've only backported it from #6129

Copy link
Copy Markdown
Collaborator

@deepthi deepthi Oct 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harshit-gangal should be ok to change the jdk version from openjdk-8 to default-jdk?

Copy link
Copy Markdown
Member

@harshit-gangal harshit-gangal Oct 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to keep it at jdk8 otherwise java code will not compile. Any reason for changing this?

Copy link
Copy Markdown
Member Author

@guidoiaquinti guidoiaquinti Oct 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to keep it at jdk8 otherwise java code will not compile. Any reason for changing this?

Unfortunately there isn't openjdk-8-jdk available for Buster.

In stretch, default-jdk points to the Java runtime, or Java compatible development kit recommended for this architecture, which is openjdk-8-jdk for amd64 (see doc).

In buster, default-jdk points to the Java runtime, or Java compatible development kit recommended for this architecture, which is openjdk-11-jdk for amd64 (see doc).

g++ \
git \
libgconf-2-4 \
libtool \
make \
openjdk-8-jdk \
maven \
software-properties-common \
virtualenv \
unzip \
xvfb \
zip \
libz-dev \
ant \
&& rm -rf /var/lib/apt/lists/*

# Install Maven 3.1+
RUN mkdir -p /vt/src/vitess.io/vitess/dist && \
cd /vt/src/vitess.io/vitess/dist && \
curl -sL --connect-timeout 10 --retry 3 \
http://www-us.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz | tar -xz && \
mv apache-maven-3.3.9 maven

# Set up Vitess environment (equivalent to '. dev.env')
ENV VTROOT /vt/src/vitess.io/vitess
ENV VTDATAROOT /vt/vtdataroot
Expand Down
2 changes: 1 addition & 1 deletion docker/bootstrap/Dockerfile.mariadb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ FROM vitess/bootstrap:common

# Install MariaDB 10
RUN for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --recv-keys 9334A25F8507EFA5 && break; done && \
add-apt-repository 'deb http://repo.percona.com/apt stretch main' && \
add-apt-repository 'deb http://repo.percona.com/apt buster main' && \
{ \
echo debconf debconf/frontend select Noninteractive; \
echo percona-server-server-5.7 percona-server-server/root_password password 'unused'; \
Expand Down
2 changes: 1 addition & 1 deletion docker/bootstrap/Dockerfile.mariadb103
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ FROM vitess/bootstrap:common

# Install MariaDB 10.3
RUN apt-key adv --no-tty --recv-keys --keyserver keyserver.ubuntu.com 0xF1656F24C74CD1D8 \
&& add-apt-repository 'deb [arch=amd64] http://ftp.osuosl.org/pub/mariadb/repo/10.3/debian stretch main' \
&& add-apt-repository 'deb [arch=amd64] http://ftp.osuosl.org/pub/mariadb/repo/10.3/debian buster main' \
&& apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
mariadb-server-10.3 \
libmariadb-dev \
Expand Down
8 changes: 7 additions & 1 deletion docker/bootstrap/Dockerfile.mysql56
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
FROM vitess/bootstrap:common

# Install MySQL 5.6
#
# Unfortunately we need to keep the 'stretch' repo from Oracle as there's no official support
# for MySQL 5.6 for Debian Buster: https://bugs.mysql.com/bug.php?id=101055
#
# I think it's fine as MySQL 5.6 will be EOL pretty soon (February 5, 2021)
#
RUN for i in $(seq 1 10); do apt-key adv --no-tty --recv-keys --keyserver pool.sks-keyservers.net 8C718D3B5072E1F5 && break; done && \
add-apt-repository 'deb http://repo.mysql.com/apt/debian/ stretch mysql-5.6' && \
for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --recv-keys 9334A25F8507EFA5 && break; done && \
echo 'deb http://repo.percona.com/apt stretch main' > /etc/apt/sources.list.d/percona.list && \
echo 'deb http://repo.percona.com/apt buster main' > /etc/apt/sources.list.d/percona.list && \
{ \
echo debconf debconf/frontend select Noninteractive; \
echo percona-server-server-5.6 percona-server-server/root_password password 'unused'; \
Expand Down
4 changes: 2 additions & 2 deletions docker/bootstrap/Dockerfile.mysql57
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ FROM vitess/bootstrap:common
# Install MySQL 5.7
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends gnupg dirmngr ca-certificates && \
for i in $(seq 1 10); do apt-key adv --no-tty --recv-keys --keyserver ha.pool.sks-keyservers.net 8C718D3B5072E1F5 && break; done && \
add-apt-repository 'deb http://repo.mysql.com/apt/debian/ stretch mysql-5.7' && \
add-apt-repository 'deb http://repo.mysql.com/apt/debian/ buster mysql-5.7' && \
for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --recv-keys 9334A25F8507EFA5 && break; done && \
echo 'deb http://repo.percona.com/apt stretch main' > /etc/apt/sources.list.d/percona.list && \
echo 'deb http://repo.percona.com/apt buster main' > /etc/apt/sources.list.d/percona.list && \
{ \
echo debconf debconf/frontend select Noninteractive; \
echo percona-server-server-5.7 percona-server-server/root_password password 'unused'; \
Expand Down
4 changes: 2 additions & 2 deletions docker/bootstrap/Dockerfile.mysql80
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ FROM vitess/bootstrap:common

# Install MySQL 8.0
RUN for i in $(seq 1 10); do apt-key adv --no-tty --recv-keys --keyserver ha.pool.sks-keyservers.net 8C718D3B5072E1F5 && break; done && \
add-apt-repository 'deb http://repo.mysql.com/apt/debian/ stretch mysql-8.0' && \
add-apt-repository 'deb http://repo.mysql.com/apt/debian/ buster mysql-8.0' && \
for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --recv-keys 9334A25F8507EFA5 && break; done && \
echo 'deb http://repo.percona.com/apt stretch main' > /etc/apt/sources.list.d/percona.list && \
echo 'deb http://repo.percona.com/apt buster main' > /etc/apt/sources.list.d/percona.list && \
{ \
echo debconf debconf/frontend select Noninteractive; \
echo percona-server-server-8.0 percona-server-server/root_password password 'unused'; \
Expand Down
14 changes: 13 additions & 1 deletion docker/bootstrap/Dockerfile.percona
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,28 @@
FROM vitess/bootstrap:common

# Install Percona 5.6
#
# Unfortunately we need to keep the 'stretch' repo from Percona as there's no official support
# for MySQL 5.6 for Debian Buster
#
# I think it's fine as MySQL 5.6 will be EOL pretty soon (February 5, 2021)
#
# Also, for the 'percona-xtrabackup-24' package we need to specificly target the
# 'buster' repository as the 'stretch' package requires 'libcurl3' that is not present
# in Debian Buster.
#
RUN for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --recv-keys 9334A25F8507EFA5 && break; done && \
add-apt-repository 'deb http://repo.percona.com/apt stretch main' && \
add-apt-repository 'deb http://repo.percona.com/apt buster main' && \
{ \
echo debconf debconf/frontend select Noninteractive; \
echo percona-server-server-5.6 percona-server-server/root_password password 'unused'; \
echo percona-server-server-5.6 percona-server-server/root_password_again password 'unused'; \
} | debconf-set-selections && \
apt-get update && \
apt-get install -y --no-install-recommends \
percona-server-server-5.6 libperconaserverclient18.1-dev rsync libev4 percona-xtrabackup-24 && \
percona-server-server-5.6 libperconaserverclient18.1-dev rsync libev4 && \
apt-get install -y -t buster percona-xtrabackup-24 && \
rm -rf /var/lib/apt/lists/*

# Bootstrap Vitess
Expand Down
4 changes: 2 additions & 2 deletions docker/bootstrap/Dockerfile.percona57
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ FROM vitess/bootstrap:common

# Install Percona 5.7
RUN for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --recv-keys 9334A25F8507EFA5 && break; done && \
add-apt-repository 'deb http://repo.percona.com/apt stretch main' && \
add-apt-repository 'deb http://repo.percona.com/apt buster main' && \
{ \
echo debconf debconf/frontend select Noninteractive; \
echo percona-server-server-5.7 percona-server-server/root_password password 'unused'; \
Expand All @@ -11,7 +11,7 @@ RUN for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --r
apt-get update && \
apt-get install -y --no-install-recommends \
percona-server-server-5.7 \
libperconaserverclient18.1-dev percona-xtrabackup-24 && \
Copy link
Copy Markdown
Member Author

@guidoiaquinti guidoiaquinti Oct 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libperconaserverclient18.1-dev doesn't exist in Percona/buster

libperconaserverclient20-dev percona-xtrabackup-24 && \
rm -rf /var/lib/apt/lists/*

# Bootstrap Vitess
Expand Down
4 changes: 2 additions & 2 deletions docker/bootstrap/Dockerfile.percona80
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ FROM vitess/bootstrap:common

# Install Percona 8.0
RUN for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --recv-keys 9334A25F8507EFA5 && break; done \
&& echo 'deb http://repo.percona.com/ps-80/apt stretch main' > /etc/apt/sources.list.d/percona.list && \
&& echo 'deb http://repo.percona.com/ps-80/apt buster main' > /etc/apt/sources.list.d/percona.list && \
{ \
echo debconf debconf/frontend select Noninteractive; \
echo percona-server-server-8.0 percona-server-server/root_password password 'unused'; \
Expand All @@ -19,7 +19,7 @@ RUN for i in $(seq 1 10); do apt-key adv --no-tty --keyserver keys.gnupg.net --r
rsync \
libev4 \
# && rm -f /etc/apt/sources.list.d/percona.list \
&& echo 'deb http://repo.percona.com/apt stretch main' > /etc/apt/sources.list.d/percona.list \
&& echo 'deb http://repo.percona.com/apt buster main' > /etc/apt/sources.list.d/percona.list \
# { \
# echo debconf debconf/frontend select Noninteractive; \
# echo percona-server-server-8.0 percona-server-server/root_password password 'unused'; \
Expand Down
2 changes: 1 addition & 1 deletion docker/lite/Dockerfile.mariadb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ USER vitess
RUN make install PREFIX=/vt/install

# Start over and build the final image.
FROM debian:stretch-slim
FROM debian:buster-slim

# Install dependencies
COPY docker/lite/install_dependencies.sh /vt/dist/install_dependencies.sh
Expand Down
2 changes: 1 addition & 1 deletion docker/lite/Dockerfile.mariadb103
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ USER vitess
RUN make install PREFIX=/vt/install

# Start over and build the final image.
FROM debian:stretch-slim
FROM debian:buster-slim

# Install dependencies
COPY docker/lite/install_dependencies.sh /vt/dist/install_dependencies.sh
Expand Down
2 changes: 1 addition & 1 deletion docker/lite/Dockerfile.mysql56
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ USER vitess
RUN make install PREFIX=/vt/install

# Start over and build the final image.
FROM debian:stretch-slim
FROM debian:buster-slim

# Install dependencies
COPY docker/lite/install_dependencies.sh /vt/dist/install_dependencies.sh
Expand Down
2 changes: 1 addition & 1 deletion docker/lite/Dockerfile.mysql57
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ USER vitess
RUN make install PREFIX=/vt/install

# Start over and build the final image.
FROM debian:stretch-slim
FROM debian:buster-slim

# Install dependencies
COPY docker/lite/install_dependencies.sh /vt/dist/install_dependencies.sh
Expand Down
2 changes: 1 addition & 1 deletion docker/lite/Dockerfile.mysql80
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ USER vitess
RUN make install PREFIX=/vt/install

# Start over and build the final image.
FROM debian:stretch-slim
FROM debian:buster-slim

# Install dependencies
COPY docker/lite/install_dependencies.sh /vt/dist/install_dependencies.sh
Expand Down
2 changes: 1 addition & 1 deletion docker/lite/Dockerfile.percona
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ USER vitess
RUN make install PREFIX=/vt/install

# Start over and build the final image.
FROM debian:stretch-slim
FROM debian:buster-slim

# Install dependencies
COPY docker/lite/install_dependencies.sh /vt/dist/install_dependencies.sh
Expand Down
2 changes: 1 addition & 1 deletion docker/lite/Dockerfile.percona57
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ USER vitess
RUN make install PREFIX=/vt/install

# Start over and build the final image.
FROM debian:stretch-slim
FROM debian:buster-slim

# Install dependencies
COPY docker/lite/install_dependencies.sh /vt/dist/install_dependencies.sh
Expand Down
2 changes: 1 addition & 1 deletion docker/lite/Dockerfile.percona80
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ USER vitess
RUN make install PREFIX=/vt/install

# Start over and build the final image.
FROM debian:stretch-slim
FROM debian:buster-slim

# Install dependencies
COPY docker/lite/install_dependencies.sh /vt/dist/install_dependencies.sh
Expand Down
2 changes: 1 addition & 1 deletion docker/lite/Dockerfile.testing
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ USER vitess
RUN make install-testing PREFIX=/vt/install

# Start over and build the final image.
FROM debian:stretch-slim
FROM debian:buster-slim

# Install dependencies
COPY docker/lite/install_dependencies.sh /vt/dist/install_dependencies.sh
Expand Down
16 changes: 8 additions & 8 deletions docker/lite/install_dependencies.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,30 +126,30 @@ add_apt_key 9334A25F8507EFA5
# Add extra apt repositories for MySQL.
case "${FLAVOR}" in
mysql56)
echo 'deb http://repo.mysql.com/apt/debian/ stretch mysql-5.6' > /etc/apt/sources.list.d/mysql.list
echo 'deb http://repo.mysql.com/apt/debian/ buster mysql-5.6' > /etc/apt/sources.list.d/mysql.list
;;
mysql57)
echo 'deb http://repo.mysql.com/apt/debian/ stretch mysql-5.7' > /etc/apt/sources.list.d/mysql.list
echo 'deb http://repo.mysql.com/apt/debian/ buster mysql-5.7' > /etc/apt/sources.list.d/mysql.list
;;
mysql80)
echo 'deb http://repo.mysql.com/apt/debian/ stretch mysql-8.0' > /etc/apt/sources.list.d/mysql.list
echo 'deb http://repo.mysql.com/apt/debian/ buster mysql-8.0' > /etc/apt/sources.list.d/mysql.list
;;
mariadb)
echo 'deb http://sfo1.mirrors.digitalocean.com/mariadb/repo/10.2/debian stretch main' > /etc/apt/sources.list.d/mariadb.list
echo 'deb http://sfo1.mirrors.digitalocean.com/mariadb/repo/10.2/debian buster main' > /etc/apt/sources.list.d/mariadb.list
;;
mariadb103)
echo 'deb http://sfo1.mirrors.digitalocean.com/mariadb/repo/10.3/debian stretch main' > /etc/apt/sources.list.d/mariadb.list
echo 'deb http://sfo1.mirrors.digitalocean.com/mariadb/repo/10.3/debian buster main' > /etc/apt/sources.list.d/mariadb.list
;;
esac

# Add extra apt repositories for Percona Server and/or Percona XtraBackup.
case "${FLAVOR}" in
mysql56|mysql57|mysql80|percona|percona57)
echo 'deb http://repo.percona.com/apt stretch main' > /etc/apt/sources.list.d/percona.list
echo 'deb http://repo.percona.com/apt buster main' > /etc/apt/sources.list.d/percona.list
;;
percona80)
echo 'deb http://repo.percona.com/apt stretch main' > /etc/apt/sources.list.d/percona.list
echo 'deb http://repo.percona.com/ps-80/apt stretch main' > /etc/apt/sources.list.d/percona80.list
echo 'deb http://repo.percona.com/apt buster main' > /etc/apt/sources.list.d/percona.list
echo 'deb http://repo.percona.com/ps-80/apt buster main' > /etc/apt/sources.list.d/percona80.list
;;
esac

Expand Down
2 changes: 1 addition & 1 deletion docker/orchestrator/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ script="go get vitess.io/vitess/go/cmd/vtctlclient && \
go install github.com/openark/orchestrator/go/cmd/orchestrator"

echo "Building orchestrator..."
docker run -ti --name=vt_orc_build golang:1.14.4-stretch bash -c "$script"
docker run -ti --name=vt_orc_build golang:1.14.4-buster bash -c "$script"
Copy link
Copy Markdown
Member Author

@guidoiaquinti guidoiaquinti Oct 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is technically not needed as orchestrator has its own build process but I think it's 👍 if we align the versions across the whole repo.

docker cp vt_orc_build:/go/bin/orchestrator $tmpdir
docker cp vt_orc_build:/go/bin/vtctlclient $tmpdir
docker cp vt_orc_build:/go/src/github.com/openark/orchestrator/resources $tmpdir
Expand Down