Skip to content

Commit 3bfb91b

Browse files
RedTopperAbdBarho
authored and
Jordan-Lambda
committed
Make Dockerfiles OCI compliant (AbdBarho#408)
## Justification Closes issue AbdBarho#352 This update makes the Dockerfiles OCI compliant, making it easier to use Buildah or other image building techniques that require it ## Implementation This changes a few things, listed below: * auto: Download container is switched to alpine. The `git` container specified the `/git` directory as a volume. As such, all the files under `/git` would be lost after each script invoke. Alpine is used later in the build process anyway, so it shouldn't be any extra cost to switch to it * auto: "New" clone.sh script is copied into the container, which is basically just the previous clone script that was embedded in the Dockerfile. * all: `<<EOF` heredoc styles have been switched to `&& \` * all: I added NVIDIA_DRIVER_CAPABILITIES and NVIDIA_VISIBLE_DEVICES to expose my Nvidia card. This is most likely a selinux/podman problem, but shouldn't change anything with docker to add it. * docker-compose: I added selinux labeling. I tested this with real docker (not just podman!) and it seems to work fine. Though I suggest you try it too. ## Testing Locally builds with buildah. Note: for caching to work properly, you still need to replace `/root/.cache/pip` with `/root/.cache/pip,Z` on selinux systems. Note: I was having some trouble running invoke. Thought it was this PR, but it's a known issue. See invoke-ai/InvokeAI#3182 --------- Co-authored-by: AbdBarho <ka70911@gmail.com>
1 parent 33076db commit 3bfb91b

File tree

4 files changed

+71
-81
lines changed

4 files changed

+71
-81
lines changed

‎services/AUTOMATIC1111/Dockerfile

+21-31
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,6 @@
1-
# syntax=docker/dockerfile:1
2-
31
FROM alpine/git:2.36.2 as download
42

5-
SHELL ["/bin/sh", "-ceuxo", "pipefail"]
6-
7-
RUN <<EOF
8-
cat <<'EOE' > /clone.sh
9-
mkdir -p repositories/"$1" && cd repositories/"$1" && git init && git remote add origin "$2" && git fetch origin "$3" --depth=1 && git reset --hard "$3" && rm -rf .git
10-
EOE
11-
EOF
3+
COPY clone.sh /clone.sh
124

135
RUN . /clone.sh taming-transformers https://github.com/CompVis/taming-transformers.git 24268930bf1dce879235a7fddd0b2355b84d7ea6 \
146
&& rm -rf data assets **/*.ipynb
@@ -30,21 +22,19 @@ RUN aria2c -x 5 --dir / --out wheel.whl 'https://github.com/AbdBarho/stable-diff
3022

3123
FROM python:3.10.9-slim
3224

33-
SHELL ["/bin/bash", "-ceuxo", "pipefail"]
34-
3525
ENV DEBIAN_FRONTEND=noninteractive PIP_PREFER_BINARY=1
3626

37-
RUN PIP_NO_CACHE_DIR=1 pip install torch==1.13.1+cu117 torchvision --extra-index-url https://download.pytorch.org/whl/cu117
27+
RUN --mount=type=cache,target=/root/.cache/pip \
28+
pip install torch==1.13.1+cu117 torchvision --extra-index-url https://download.pytorch.org/whl/cu117
3829

3930
RUN apt-get update && apt install fonts-dejavu-core rsync git jq moreutils -y && apt-get clean
4031

4132

42-
RUN --mount=type=cache,target=/root/.cache/pip <<EOF
43-
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
44-
cd stable-diffusion-webui
45-
git reset --hard d7aec59c4eb02f723b3d55c6f927a42e97acd679
46-
pip install -r requirements_versions.txt
47-
EOF
33+
RUN --mount=type=cache,target=/root/.cache/pip \
34+
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git && \
35+
cd stable-diffusion-webui && \
36+
git reset --hard d7aec59c4eb02f723b3d55c6f927a42e97acd679 && \
37+
pip install -r requirements_versions.txt
4838

4939
RUN --mount=type=cache,target=/root/.cache/pip \
5040
--mount=type=bind,from=xformers,source=/wheel.whl,target=/xformers-0.0.15-cp310-cp310-linux_x86_64.whl \
@@ -53,7 +43,7 @@ RUN --mount=type=cache,target=/root/.cache/pip \
5343
ENV ROOT=/stable-diffusion-webui
5444

5545

56-
COPY --from=download /git/ ${ROOT}
46+
COPY --from=download /repositories/ ${ROOT}/repositories/
5747
RUN mkdir ${ROOT}/interrogate && cp ${ROOT}/repositories/clip-interrogator/data/* ${ROOT}/interrogate
5848
RUN --mount=type=cache,target=/root/.cache/pip \
5949
pip install -r ${ROOT}/repositories/CodeFormer/requirements.txt
@@ -72,25 +62,25 @@ RUN apt-get -y install libgoogle-perftools-dev && apt-get clean
7262
ENV LD_PRELOAD=libtcmalloc.so
7363

7464
ARG SHA=a9fed7c364061ae6efb37f797b6b522cb3cf7aa2
75-
RUN --mount=type=cache,target=/root/.cache/pip <<EOF
76-
cd stable-diffusion-webui
77-
git fetch
78-
git reset --hard ${SHA}
79-
pip install -r requirements_versions.txt
80-
EOF
65+
RUN --mount=type=cache,target=/root/.cache/pip \
66+
cd stable-diffusion-webui && \
67+
git fetch && \
68+
git reset --hard ${SHA} && \
69+
pip install -r requirements_versions.txt
8170

8271
RUN --mount=type=cache,target=/root/.cache/pip pip install -U opencv-python-headless
8372

8473
COPY . /docker
8574

86-
RUN <<EOF
87-
python3 /docker/info.py ${ROOT}/modules/ui.py
88-
mv ${ROOT}/style.css ${ROOT}/user.css
89-
# one of the ugliest hacks I ever wrote
90-
sed -i 's/in_app_dir = .*/in_app_dir = True/g' /usr/local/lib/python3.10/site-packages/gradio/routes.py
91-
EOF
75+
RUN \
76+
python3 /docker/info.py ${ROOT}/modules/ui.py && \
77+
mv ${ROOT}/style.css ${ROOT}/user.css && \
78+
# one of the ugliest hacks I ever wrote \
79+
sed -i 's/in_app_dir = .*/in_app_dir = True/g' /usr/local/lib/python3.10/site-packages/gradio/routes.py
9280

9381
WORKDIR ${ROOT}
82+
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
83+
ENV NVIDIA_VISIBLE_DEVICES=all
9484
ENV CLI_ARGS=""
9585
EXPOSE 7860
9686
ENTRYPOINT ["/docker/entrypoint.sh"]

‎services/AUTOMATIC1111/clone.sh

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
#!/bin/bash
2+
3+
set -Eeuox pipefail
4+
5+
mkdir -p /repositories/"$1"
6+
cd /repositories/"$1"
7+
git init
8+
git remote add origin "$2"
9+
git fetch origin "$3" --depth=1
10+
git reset --hard "$3"
11+
rm -rf .git

‎services/invoke/Dockerfile

+21-26
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,10 @@
1-
# syntax=docker/dockerfile:1
2-
31
FROM alpine:3.17 as xformers
42
RUN apk add --no-cache aria2
53
RUN aria2c -x 5 --dir / --out wheel.whl 'https://github.com/AbdBarho/stable-diffusion-webui-docker/releases/download/5.0.0/xformers-0.0.17.dev449-cp310-cp310-manylinux2014_x86_64.whl'
64

75

86

97
FROM python:3.10-slim
10-
SHELL ["/bin/bash", "-ceuxo", "pipefail"]
118

129
ENV DEBIAN_FRONTEND=noninteractive PIP_EXISTS_ACTION=w PIP_PREFER_BINARY=1
1310

@@ -20,35 +17,32 @@ RUN git clone https://github.com/invoke-ai/InvokeAI.git /stable-diffusion
2017

2118
WORKDIR /stable-diffusion
2219

23-
RUN --mount=type=cache,target=/root/.cache/pip <<EOF
24-
git reset --hard f232068ab89bd80e4f5f3133dcdb62ea78f1d0f7
25-
git config --global http.postBuffer 1048576000
26-
egrep -v '^-e .' environments-and-requirements/requirements-lin-cuda.txt > req.txt
27-
pip install -r req.txt
28-
rm req.txt
29-
EOF
20+
RUN --mount=type=cache,target=/root/.cache/pip \
21+
git reset --hard f232068ab89bd80e4f5f3133dcdb62ea78f1d0f7 && \
22+
git config --global http.postBuffer 1048576000 && \
23+
egrep -v '^-e .' environments-and-requirements/requirements-lin-cuda.txt > req.txt && \
24+
pip install -r req.txt && \
25+
rm req.txt
3026

3127

3228
# patch match:
3329
# https://github.com/invoke-ai/InvokeAI/blob/main/docs/installation/INSTALL_PATCHMATCH.md
34-
RUN <<EOF
35-
apt-get update
36-
# apt-get install build-essential python3-opencv libopencv-dev -y
37-
apt-get install make g++ libopencv-dev -y
38-
apt-get clean
39-
cd /usr/lib/x86_64-linux-gnu/pkgconfig/
40-
ln -sf opencv4.pc opencv.pc
41-
EOF
30+
RUN \
31+
apt-get update && \
32+
# apt-get install build-essential python3-opencv libopencv-dev -y && \
33+
apt-get install make g++ libopencv-dev -y && \
34+
apt-get clean && \
35+
cd /usr/lib/x86_64-linux-gnu/pkgconfig/ && \
36+
ln -sf opencv4.pc opencv.pc
4237

4338

4439
ARG BRANCH=main SHA=6e0c6d9cc9f6bdbdefc4b9e94bc1ccde1b04aa42
45-
RUN --mount=type=cache,target=/root/.cache/pip <<EOF
46-
git fetch
47-
git reset --hard
48-
git checkout ${BRANCH}
49-
git reset --hard ${SHA}
50-
pip install .
51-
EOF
40+
RUN --mount=type=cache,target=/root/.cache/pip \
41+
git fetch && \
42+
git reset --hard && \
43+
git checkout ${BRANCH} && \
44+
git reset --hard ${SHA} && \
45+
pip install .
5246

5347

5448
RUN --mount=type=cache,target=/root/.cache/pip \
@@ -60,7 +54,8 @@ RUN --mount=type=cache,target=/root/.cache/pip \
6054
RUN touch invokeai.init
6155
COPY . /docker/
6256

63-
57+
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
58+
ENV NVIDIA_VISIBLE_DEVICES=all
6459
ENV PYTHONUNBUFFERED=1 ROOT=/stable-diffusion PYTHONPATH="${PYTHONPATH}:${ROOT}" PRELOAD=false CLI_ARGS="" HF_HOME=/root/.cache/huggingface
6560
EXPOSE 7860
6661

‎services/sygil/Dockerfile

+18-24
Original file line numberDiff line numberDiff line change
@@ -1,45 +1,39 @@
1-
# syntax=docker/dockerfile:1
2-
31
FROM python:3.8-slim
42

5-
SHELL ["/bin/bash", "-ceuxo", "pipefail"]
6-
73
ENV DEBIAN_FRONTEND=noninteractive PIP_PREFER_BINARY=1
84

95
RUN --mount=type=cache,target=/root/.cache/pip pip install torch==1.13.0 torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu117
106

117
RUN apt-get update && apt install gcc libsndfile1 ffmpeg build-essential zip unzip git -y && apt-get clean
128

13-
RUN --mount=type=cache,target=/root/.cache/pip <<EOF
14-
git config --global http.postBuffer 1048576000
15-
git clone https://github.com/Sygil-Dev/sygil-webui.git stable-diffusion
16-
cd stable-diffusion
17-
git reset --hard 5291437085bddd16d752f811b6552419a2044d12
18-
pip install -r requirements.txt
19-
EOF
9+
RUN --mount=type=cache,target=/root/.cache/pip \
10+
git config --global http.postBuffer 1048576000 && \
11+
git clone https://github.com/Sygil-Dev/sygil-webui.git stable-diffusion && \
12+
cd stable-diffusion && \
13+
git reset --hard 5291437085bddd16d752f811b6552419a2044d12 && \
14+
pip install -r requirements.txt
2015

2116

2217
ARG BRANCH=master SHA=571fb897edd58b714bb385dfaa1ad59aecef8bc7
23-
RUN --mount=type=cache,target=/root/.cache/pip <<EOF
24-
cd stable-diffusion
25-
git fetch
26-
git checkout ${BRANCH}
27-
git reset --hard ${SHA}
28-
pip install -r requirements.txt
29-
EOF
18+
RUN --mount=type=cache,target=/root/.cache/pip \
19+
cd stable-diffusion && \
20+
git fetch && \
21+
git checkout ${BRANCH} && \
22+
git reset --hard ${SHA} && \
23+
pip install -r requirements.txt
3024

3125
RUN --mount=type=cache,target=/root/.cache/pip pip install -U 'transformers>=4.24'
3226

3327
# add info
3428
COPY . /docker/
35-
RUN <<EOF
36-
python /docker/info.py /stable-diffusion/frontend/frontend.py
37-
chmod +x /docker/mount.sh /docker/run.sh
38-
# streamlit
39-
sed -i -- 's/8501/7860/g' /stable-diffusion/.streamlit/config.toml
40-
EOF
29+
RUN python /docker/info.py /stable-diffusion/frontend/frontend.py && \
30+
chmod +x /docker/mount.sh /docker/run.sh && \
31+
# streamlit \
32+
sed -i -- 's/8501/7860/g' /stable-diffusion/.streamlit/config.toml
4133

4234
WORKDIR /stable-diffusion
35+
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
36+
ENV NVIDIA_VISIBLE_DEVICES=all
4337
ENV PYTHONPATH="${PYTHONPATH}:${PWD}" STREAMLIT_SERVER_HEADLESS=true USE_STREAMLIT=0 CLI_ARGS=""
4438
EXPOSE 7860
4539

0 commit comments

Comments
 (0)