Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dev: reduce docker layers & combine install commands #992

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

dev: reduce docker layers & combine install commands #992

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 10 additions & 91 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,103 +17,25 @@
FROM ubuntu:22.04
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && apt-get upgrade -y && apt-get install -y wget nano rsync curl gnupg2 jq unzip bzip2

# for clang-*-15, see https://apt.llvm.org/
RUN echo "deb http://apt.llvm.org/jammy/ llvm-toolchain-jammy-15 main" >> /etc/apt/sources.list && \
echo "deb-src http://apt.llvm.org/jammy/ llvm-toolchain-jammy-15 main" >> /etc/apt/sources.list && \
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add -

# Install gcc8-arm-none-eabi
RUN mkdir ~/Downloads &&\
cd ~/Downloads &&\
wget -O gcc.tar.bz2 https://developer.arm.com/-/media/Files/downloads/gnu-rm/8-2018q4/gcc-arm-none-eabi-8-2018-q4-major-linux.tar.bz2?revision=d830f9dd-cd4f-406d-8672-cca9210dd220?product=GNU%20Arm%20Embedded%20Toolchain,64-bit,,Linux,8-2018-q4-major &&\
echo "fb31fbdfe08406ece43eef5df623c0b2deb8b53e405e2c878300f7a1f303ee52 gcc.tar.bz2" | sha256sum -c &&\
cd ~/Downloads &&\
tar -xjvf gcc.tar.bz2 &&\
rm -f gcc.tar.bz2 &&\
cd ~/Downloads && rsync -a gcc-arm-none-eabi-8-2018-q4-major/ /usr/local/

# Tools for building
RUN apt-get update && apt-get install -y \
build-essential \
llvm-15 \
gcc-10 \
binutils \
valgrind \
cmake \
git \
autotools-dev \
automake \
autoconf \
libtool \
pkg-config \
libcmocka-dev \
libc6-i386 \
lib32stdc++6 \
lib32z1 \
libusb-1.0-0-dev \
libudev-dev \
libhidapi-dev

RUN apt-get update && apt-get install -y \
doxygen \
graphviz

# Set gcc-10 as the default gcc
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100
RUN update-alternatives --install /usr/bin/gcov gcov /usr/bin/gcov-10 100

# Tools for CI
RUN apt-get update && apt-get install -y \
python3 \
python3-pip \
clang-format-15 \
clang-tidy-15

RUN python3 -m pip install --upgrade pip
ADD scripts/docker_install.sh /tmp/
RUN chmod +x /tmp/docker_install.sh && /tmp/docker_install.sh

# Python modules
COPY py/bitbox02 /tmp/bitbox02
RUN python3 -m pip install /tmp/bitbox02
RUN rm -r /tmp/bitbox02
RUN python3 -m pip install /tmp/bitbox02 && \
rm -r /tmp/bitbox02
COPY py/requirements.txt /tmp
RUN python3 -m pip install --upgrade --requirement /tmp/requirements.txt
RUN rm /tmp/requirements.txt

# Python modules for CI
RUN python3 -m pip install --upgrade \
pylint==2.13.9 \
pylint-protobuf==0.20.2 \
black==22.3.0 \
mypy==0.960 \
mypy-protobuf==3.2.0

# Python modules for packaging
RUN python3 -m pip install --upgrade \
setuptools==41.2.0 \
wheel==0.33.6 \
twine==1.15.0
RUN python3 -m pip install --upgrade --requirement /tmp/requirements.txt && \
rm /tmp/requirements.txt

#Install protoc from release, because the version available on the repo is too old
RUN mkdir -p /opt/protoc && \
curl -L0 https://github.com/protocolbuffers/protobuf/releases/download/v21.2/protoc-21.2-linux-x86_64.zip -o /tmp/protoc-21.2-linux-x86_64.zip && \
unzip /tmp/protoc-21.2-linux-x86_64.zip -d /opt/protoc
ENV PATH /opt/protoc/bin:$PATH

# Make Python3 the default
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 1

# Developer tools
RUN apt-get update && apt-get install -y \
bash-completion
# Install gcovr from PIP to get a newer version than in apt repositories
RUN python3 -m pip install gcovr

# Install Go, used for the tools in tools/go and for test/gounittest
ENV GOPATH /opt/go
ENV GOROOT /opt/go_dist/go
ENV PATH $GOROOT/bin:$GOPATH/bin:$PATH
ENV GOPATH=/opt/go GOROOT=/opt/go_dist/go PATH=$GOROOT/bin:$GOPATH/bin:$PATH
RUN mkdir -p /opt/go_dist && \
curl https://dl.google.com/go/go1.19.3.linux-amd64.tar.gz | tar -xz -C /opt/go_dist

Expand All @@ -122,15 +44,12 @@ RUN cd /opt && wget https://github.com/linux-test-project/lcov/releases/download
ENV PATH /opt/lcov-1.14/bin:$PATH

# Install rust compiler
ENV PATH /opt/cargo/bin:$PATH
ENV RUSTUP_HOME=/opt/rustup
ENV PATH=/opt/cargo/bin:$PATH RUSTUP_HOME=/opt/rustup
COPY src/rust/rust-toolchain /tmp/rust-toolchain
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | CARGO_HOME=/opt/cargo sh -s -- --default-toolchain $(cat /tmp/rust-toolchain | tr -d '\r\n\t') -y
RUN rustup target add thumbv7em-none-eabi
RUN rustup component add rustfmt
RUN rustup component add clippy
RUN CARGO_HOME=/opt/cargo cargo install cbindgen --version 0.24.3
RUN CARGO_HOME=/opt/cargo cargo install bindgen-cli --version 0.61.0
RUN rustup component add rustfmt clippy
RUN CARGO_HOME=/opt/cargo cargo install [email protected] [email protected]

COPY tools/prost-build prost-build
RUN CARGO_HOME=/opt/cargo cargo install --path prost-build --locked
Expand Down
76 changes: 76 additions & 0 deletions scripts/docker_install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
#!/bin/bash

apt-get update
apt-get upgrade -y
apt-get install -y wget nano rsync curl gnupg2 jq unzip bzip2

# for clang-*-15, see https://apt.llvm.org/
echo "deb http://apt.llvm.org/jammy/ llvm-toolchain-jammy-15 main" >> /etc/apt/sources.list
echo "deb-src http://apt.llvm.org/jammy/ llvm-toolchain-jammy-15 main" >> /etc/apt/sources.list
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add -

# Install gcc8-arm-none-eabi
mkdir ~/Downloads
cd ~/Downloads
wget -O gcc.tar.bz2 https://developer.arm.com/-/media/Files/downloads/gnu-rm/8-2018q4/gcc-arm-none-eabi-8-2018-q4-major-linux.tar.bz2?revision=d830f9dd-cd4f-406d-8672-cca9210dd220?product=GNU%20Arm%20Embedded%20Toolchain,64-bit,,Linux,8-2018-q4-major
echo "fb31fbdfe08406ece43eef5df623c0b2deb8b53e405e2c878300f7a1f303ee52 gcc.tar.bz2" | sha256sum -c
cd ~/Downloads
tar -xjvf gcc.tar.bz2
rm -f gcc.tar.bz2
cd ~/Downloads && rsync -a gcc-arm-none-eabi-8-2018-q4-major/ /usr/local/

# Tools for building
apt-get update
apt-get install -y \
build-essential \
llvm-15 \
gcc-10 \
binutils \
valgrind \
cmake \
git \
autotools-dev \
automake \
autoconf \
libtool \
pkg-config \
libcmocka-dev \
libc6-i386 \
lib32stdc++6 \
lib32z1 \
libusb-1.0-0-dev \
libudev-dev \
libhidapi-dev \
doxygen \
graphviz \
python3 \
python3-pip \
clang-format-15 \
clang-tidy-15 \
Comment on lines +44 to +49
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These were in separate RUNs before grouped by topic (docs, CI). Would be nice to to add a comment before each group like you did below with the pip installs.

bash-completion


# Set gcc-10 as the default gcc
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100
update-alternatives --install /usr/bin/gcov gcov /usr/bin/gcov-10 100

# Make Python3 the default
update-alternatives --install /usr/bin/python python /usr/bin/python3 1

# install pip
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upgrade pip - it was installed before

python -m pip install --upgrade pip

python -m pip install --upgrade \
# Python modules for CI
pylint==2.13.9 \
pylint-protobuf==0.20.2 \
black==22.3.0 \
mypy==0.960 \
mypy-protobuf==3.2.0 \
# Python modules for packaging
setuptools==41.2.0 \
wheel==0.33.6 \
twine==1.15.0

# Install gcovr from PIP to get a newer version than in apt repositories
python -m pip install gcovr