Skip to content

[CI] Disable pip cache when creating Docker images #8575

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
areusch merged 1 commit into from
Jul 29, 2021
Merged

[CI] Disable pip cache when creating Docker images #8575

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docker/Dockerfile.ci_arm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ FROM ubuntu:18.04
RUN apt-get update --fix-missing
RUN apt-get install -y ca-certificates gnupg2

# Globally disable pip cache
RUN pip config set global.cache-dir false
Copy link
Contributor

@areusch areusch Jul 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it global.no-cache-dir or global.cache-dir?

Copy link
Contributor Author

@leandron leandron Jul 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it locally and it seems to work to set the global config and preventing the cache to be created.

When using it in an individual pip install —no-cache-dir foo it seems to be the way to disable in one instance, but I decided not going this way, because there is always a new pip install in those scripts.


COPY install/ubuntu_install_core.sh /install/ubuntu_install_core.sh
RUN bash /install/ubuntu_install_core.sh

Expand Down
3 changes: 3 additions & 0 deletions docker/Dockerfile.ci_cpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ FROM ubuntu:18.04

RUN apt-get update --fix-missing

# Globally disable pip cache
RUN pip config set global.cache-dir false

COPY install/ubuntu_install_core.sh /install/ubuntu_install_core.sh
RUN bash /install/ubuntu_install_core.sh

Expand Down
3 changes: 3 additions & 0 deletions docker/Dockerfile.ci_gpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ FROM nvidia/cuda:11.0.3-cudnn8-devel-ubuntu18.04
# Base scripts
RUN apt-get update --fix-missing

# Globally disable pip cache
RUN pip config set global.cache-dir false

COPY install/ubuntu_install_core.sh /install/ubuntu_install_core.sh
RUN bash /install/ubuntu_install_core.sh

Expand Down
3 changes: 3 additions & 0 deletions docker/Dockerfile.ci_i386
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ FROM ioft/i386-ubuntu:16.04

RUN apt-get update --fix-missing && apt-get install -y ca-certificates

# Globally disable pip cache
RUN pip config set global.cache-dir false

COPY install/ubuntu_install_core.sh /install/ubuntu_install_core.sh
RUN bash /install/ubuntu_install_core.sh

Expand Down
3 changes: 3 additions & 0 deletions docker/Dockerfile.ci_lint
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ FROM ubuntu:18.04

RUN apt-get update --fix-missing

# Globally disable pip cache
RUN pip config set global.cache-dir false

RUN apt-get update && apt-get install -y wget git sudo make

COPY install/ubuntu1804_install_python.sh /install/ubuntu1804_install_python.sh
Expand Down
3 changes: 3 additions & 0 deletions docker/Dockerfile.ci_qemu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ FROM ubuntu:18.04

RUN apt-get update --fix-missing

# Globally disable pip cache
RUN pip config set global.cache-dir false

COPY install/ubuntu_install_core.sh /install/ubuntu_install_core.sh
RUN bash /install/ubuntu_install_core.sh

Expand Down
3 changes: 3 additions & 0 deletions docker/Dockerfile.ci_wasm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ FROM ubuntu:18.04

RUN apt-get update --fix-missing

# Globally disable pip cache
RUN pip config set global.cache-dir false

COPY install/ubuntu_install_core.sh /install/ubuntu_install_core.sh
RUN bash /install/ubuntu_install_core.sh

Expand Down
5 changes: 4 additions & 1 deletion docker/install/ubuntu_install_darknet.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,7 @@ set -o pipefail
#install the necessary dependancies, cffi, opencv
wget -q 'https://github.com/siju-samuel/darknet/blob/master/lib/libdarknet.so?raw=true' -O libdarknet.so
debian_version=`cat /etc/debian_version`
pip3 install opencv-python cffi

pip3 install \
cffi \
opencv-python
9 changes: 6 additions & 3 deletions docker/install/ubuntu_install_onnx.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,14 @@ set -o pipefail

# We need to fix the onnx version because changing versions tends to break tests
# TODO(mbrookhart): periodically update
pip3 install onnx==1.8.1
pip3 install onnxruntime==1.7.0
pip3 install \
onnx==1.8.1 \
onnxruntime==1.7.0

# torch depends on a number of other packages, but unhelpfully, does
# not expose that in the wheel!!!
pip3 install future

pip3 install torch==1.7.0 torchvision==0.8.1
pip3 install \
torch==1.7.0 \
torchvision==0.8.1
19 changes: 18 additions & 1 deletion docker/install/ubuntu_install_python_package.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,21 @@ set -u
set -o pipefail

# install libraries for python package on ubuntu
pip3 install six numpy pytest cython decorator scipy tornado pytest pytest-xdist pytest-profiling mypy orderedset attrs requests Pillow packaging cloudpickle synr
pip3 install \
attrs \
cloudpickle \
cython \
decorator \
mypy \
numpy \
orderedset \
packaging \
Pillow \
pytest \
pytest-profiling \
pytest-xdist \
requests \
scipy \
six \
synr \
tornado
4 changes: 3 additions & 1 deletion docker/install/ubuntu_install_redis.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,6 @@ set -u
set -o pipefail

apt-get update && apt-get install -y redis-server
pip3 install "xgboost>=1.1.0" psutil
pip3 install \
psutil \
"xgboost>=1.1.0"
11 changes: 10 additions & 1 deletion docker/install/ubuntu_install_sphinx.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,13 @@ set -u
set -o pipefail

# NOTE: install docutils < 0.17 to work around https://github.com/readthedocs/sphinx_rtd_theme/issues/1115
pip3 install sphinx sphinx-gallery==0.4.0 autodocsumm sphinx_rtd_theme sphinx_autodoc_annotation matplotlib Image "commonmark>=0.7.3" "docutils>=0.11,<0.17"
pip3 install \
autodocsumm \
"commonmark>=0.7.3" \
"docutils>=0.11,<0.17" \
Image \
matplotlib \
sphinx \
sphinx_autodoc_annotation \
sphinx-gallery==0.4.0 \
sphinx_rtd_theme
5 changes: 4 additions & 1 deletion docker/install/ubuntu_install_tensorflow.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,7 @@ set -e
set -u
set -o pipefail

pip3 install tensorflow==2.4.2 keras==2.4.3 "h5py<3.0"
pip3 install \
"h5py<3.0" \
keras==2.4.3 \
tensorflow==2.4.2