track 3 docker environment error #100
Comments
|
Yeah, getting the docker environment can be tricky. When I get a chance
I'll look at it this week asap.
…On Mon, Mar 27, 2023 at 5:53 PM gary88christie ***@***.***> wrote:
Track 1 & Track 2 can run successfully.
But I met a Track 3 docker evironment error because
nvidia/cuda:11.0-devel-ubuntu18.04 is no longer available.
Install Status:
{'faiss_t3': 'fail'}
=> ERROR [internal] load metadata for docker.io/nvidia/cuda:11.0-devel-ubuntu18.04 1.6s
------
> [internal] load metadata for docker.io/nvidia/cuda:11.0-devel-ubuntu18.04:
------
Dockerfile:2
--------------------
1 |
2 | >>> FROM nvidia/cuda:11.0-devel-ubuntu18.04
3 |
4 | ENV PATH="/root/miniconda3/bin:${PATH}"
--------------------
ERROR: failed to solve: nvidia/cuda:11.0-devel-ubuntu18.04: docker.io/nvidia/cuda:11.0-devel-ubuntu18.04: not found
So I tried with a similar nvidia docker version
nvidia/cuda:11.0.3-devel-ubuntu18.04, but seems to cause a lot of package
conflicts:
Examining conflict for libgcc-ng libgomp _openmp_mutex: 67%|██████▋ | 34/51 [02:32<00:21, 1.25s/it] failed
#0 358.3
#0 358.3 UnsatisfiableError: The following specifications were found to be incompatible with a past
#0 358.3 explicit spec that is not an explicit spec in this operation (setuptools):
#0 358.3
#0 358.3 - faiss-gpu -> numpy[version='>=1.11,<2'] -> python[version='>=3.10,<3.11.0a0|>=3.11,<3.12.0a0']
#0 358.3 - faiss-gpu -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.9,<3.10.0a0|>=3.8,<3.9.0a0|>=3.5,<3.6.0a0']
#0 358.3 - python=3.6.9 -> pip -> setuptools
#0 358.3 - python=3.6.9 -> pip -> wheel
#0 358.3
#0 358.3 The following specifications were found to be incompatible with each other:
#0 358.3
#0 358.3 Output in format: Requested package -> Available versions
#0 358.3
#0 358.3 Package ncurses conflicts for:
#0 358.3 pip -> python[version='>=3.11,<3.12.0a0'] -> ncurses[version='6.0.*|>=6.0,<7.0a0|>=6.1,<7.0a0|>=6.2,<7.0a0|>=6.3,<7.0a0|>=6.4,<7.0a0']
#0 358.3 cffi -> python[version='>=3.11,<3.12.0a0'] -> ncurses[version='6.0.*|>=6.0,<7.0a0|>=6.1,<7.0a0|>=6.2,<7.0a0|>=6.3,<7.0a0|>=6.4,<7.0a0']
#0 358.3 cryptography -> python[version='>=3.11,<3.12.0a0'] -> ncurses[version='6.0.*|>=6.0,<7.0a0|>=6.1,<7.0a0|>=6.2,<7.0a0|>=6.3,<7.0a0|>=6.4,<7.0a0']
#0 358.3 pyopenssl -> python[version='>=3.11,<3.12.0a0'] -> ncurses[version='6.0.*|>=6.0,<7.0a0|>=6.1,<7.0a0|>=6.2,<7.0a0|>=6.3,<7.0a0|>=6.4,<7.0a0']
#0 358.3 brotlipyThe following specifications were found to be incompatible with your system:
#0 358.3
#0 358.3 - feature:/linux-64::__glibc==2.27=0
#0 358.3 - brotlipy -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
#0 358.3 - bzip2 -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
#0 358.3 - cffi -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3 - conda-package-handling -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3 - cryptography -> libgcc-ng -> __glibc[version='>=2.17']
#0 358.3 - cudatoolkit=11.0 -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
#0 358.3 - faiss-gpu -> libgcc-ng[version='>=8.4.0'] -> __glibc[version='>=2.17']
#0 358.3 - libffi -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3 - libgcc-ng -> __glibc[version='>=2.17']
#0 358.3 - libstdcxx-ng -> __glibc[version='>=2.17']
#0 358.3 - libuuid -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3 - ncurses -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3 - openssl -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
#0 358.3 - pycosat -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3 - python=3.6.9 -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
#0 358.3 - readline -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3 - ruamel.yaml -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3 - ruamel.yaml.clib -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3 - sqlite -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3 - tk -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
#0 358.3 - xz -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3 - zlib -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3 - zstandard -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
#0 358.3
#0 358.3 Your installed version is: 2.27
#0 358.3
#0 358.3
------
Dockerfile:11
--------------------
10 |
11 | >>> RUN wget \
12 | >>> https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \
13 | >>> && mkdir /root/.conda \
14 | >>> && bash Miniconda3-latest-Linux-x86_64.sh -b \
15 | >>> && rm -f Miniconda3-latest-Linux-x86_64.sh \
16 | >>> && conda --version \
17 | >>> && conda install -c pytorch python=3.6.9 faiss-gpu cudatoolkit=11.0
18 |
--------------------
ERROR: failed to solve: process "/bin/sh -c wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && mkdir /root/.conda && bash Miniconda3-latest-Linux-x86_64.sh -b && rm -f Miniconda3-latest-Linux-x86_64.sh && conda --version && conda install -c pytorch python=3.6.9 faiss-gpu cudatoolkit=11.0" did not complete successfully: exit code: 1
—
Reply to this email directly, view it on GitHub
<#100>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AADL6CPTWDSY7WJ2GEW232LW6IZANANCNFSM6AAAAAAWJ3IYNA>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
"gary88christie"
I'm not sure how to add you to the pull request, but please check this out
and let me know if it resolves your issue.
#101
As I mentioned before, I found getting docker to play nicely with CUDA/GPU
more challenging than it has been fun 😀. Let me know if you run into any
issues.
George
On Mon, Mar 27, 2023 at 9:09 PM George Williams ***@***.***>
wrote:
… Yeah, getting the docker environment can be tricky. When I get a chance
I'll look at it this week asap.
On Mon, Mar 27, 2023 at 5:53 PM gary88christie ***@***.***>
wrote:
> Track 1 & Track 2 can run successfully.
> But I met a Track 3 docker evironment error because
> nvidia/cuda:11.0-devel-ubuntu18.04 is no longer available.
>
> Install Status:
> {'faiss_t3': 'fail'}
>
> => ERROR [internal] load metadata for docker.io/nvidia/cuda:11.0-devel-ubuntu18.04 1.6s
> ------
> > [internal] load metadata for docker.io/nvidia/cuda:11.0-devel-ubuntu18.04:
> ------
> Dockerfile:2
> --------------------
> 1 |
> 2 | >>> FROM nvidia/cuda:11.0-devel-ubuntu18.04
> 3 |
> 4 | ENV PATH="/root/miniconda3/bin:${PATH}"
> --------------------
> ERROR: failed to solve: nvidia/cuda:11.0-devel-ubuntu18.04: docker.io/nvidia/cuda:11.0-devel-ubuntu18.04: not found
>
> So I tried with a similar nvidia docker version
> nvidia/cuda:11.0.3-devel-ubuntu18.04, but seems to cause a lot of package
> conflicts:
>
> Examining conflict for libgcc-ng libgomp _openmp_mutex: 67%|██████▋ | 34/51 [02:32<00:21, 1.25s/it] failed
> #0 358.3
> #0 358.3 UnsatisfiableError: The following specifications were found to be incompatible with a past
> #0 358.3 explicit spec that is not an explicit spec in this operation (setuptools):
> #0 358.3
> #0 358.3 - faiss-gpu -> numpy[version='>=1.11,<2'] -> python[version='>=3.10,<3.11.0a0|>=3.11,<3.12.0a0']
> #0 358.3 - faiss-gpu -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.9,<3.10.0a0|>=3.8,<3.9.0a0|>=3.5,<3.6.0a0']
> #0 358.3 - python=3.6.9 -> pip -> setuptools
> #0 358.3 - python=3.6.9 -> pip -> wheel
> #0 358.3
> #0 358.3 The following specifications were found to be incompatible with each other:
> #0 358.3
> #0 358.3 Output in format: Requested package -> Available versions
> #0 358.3
> #0 358.3 Package ncurses conflicts for:
> #0 358.3 pip -> python[version='>=3.11,<3.12.0a0'] -> ncurses[version='6.0.*|>=6.0,<7.0a0|>=6.1,<7.0a0|>=6.2,<7.0a0|>=6.3,<7.0a0|>=6.4,<7.0a0']
> #0 358.3 cffi -> python[version='>=3.11,<3.12.0a0'] -> ncurses[version='6.0.*|>=6.0,<7.0a0|>=6.1,<7.0a0|>=6.2,<7.0a0|>=6.3,<7.0a0|>=6.4,<7.0a0']
> #0 358.3 cryptography -> python[version='>=3.11,<3.12.0a0'] -> ncurses[version='6.0.*|>=6.0,<7.0a0|>=6.1,<7.0a0|>=6.2,<7.0a0|>=6.3,<7.0a0|>=6.4,<7.0a0']
> #0 358.3 pyopenssl -> python[version='>=3.11,<3.12.0a0'] -> ncurses[version='6.0.*|>=6.0,<7.0a0|>=6.1,<7.0a0|>=6.2,<7.0a0|>=6.3,<7.0a0|>=6.4,<7.0a0']
>
>
> #0 358.3 brotlipyThe following specifications were found to be incompatible with your system:
> #0 358.3
> #0 358.3 - feature:/linux-64::__glibc==2.27=0
> #0 358.3 - brotlipy -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
> #0 358.3 - bzip2 -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
> #0 358.3 - cffi -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3 - conda-package-handling -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3 - cryptography -> libgcc-ng -> __glibc[version='>=2.17']
> #0 358.3 - cudatoolkit=11.0 -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
> #0 358.3 - faiss-gpu -> libgcc-ng[version='>=8.4.0'] -> __glibc[version='>=2.17']
> #0 358.3 - libffi -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3 - libgcc-ng -> __glibc[version='>=2.17']
> #0 358.3 - libstdcxx-ng -> __glibc[version='>=2.17']
> #0 358.3 - libuuid -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3 - ncurses -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3 - openssl -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
> #0 358.3 - pycosat -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3 - python=3.6.9 -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
> #0 358.3 - readline -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3 - ruamel.yaml -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3 - ruamel.yaml.clib -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3 - sqlite -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3 - tk -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
> #0 358.3 - xz -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3 - zlib -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3 - zstandard -> libgcc-ng[version='>=11.2.0'] -> __glibc[version='>=2.17']
> #0 358.3
> #0 358.3 Your installed version is: 2.27
> #0 358.3
> #0 358.3
> ------
> Dockerfile:11
> --------------------
> 10 |
> 11 | >>> RUN wget \
> 12 | >>> https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \
> 13 | >>> && mkdir /root/.conda \
> 14 | >>> && bash Miniconda3-latest-Linux-x86_64.sh -b \
> 15 | >>> && rm -f Miniconda3-latest-Linux-x86_64.sh \
> 16 | >>> && conda --version \
> 17 | >>> && conda install -c pytorch python=3.6.9 faiss-gpu cudatoolkit=11.0
> 18 |
> --------------------
> ERROR: failed to solve: process "/bin/sh -c wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && mkdir /root/.conda && bash Miniconda3-latest-Linux-x86_64.sh -b && rm -f Miniconda3-latest-Linux-x86_64.sh && conda --version && conda install -c pytorch python=3.6.9 faiss-gpu cudatoolkit=11.0" did not complete successfully: exit code: 1
>
> —
> Reply to this email directly, view it on GitHub
> <#100>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AADL6CPTWDSY7WJ2GEW232LW6IZANANCNFSM6AAAAAAWJ3IYNA>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>
|
Thank you for your quick action and sorry for the late reply. And is my modification of dockerfile which changed python to 3.10 and docker to nvidia/cuda:11.0.3-devel-ubuntu18.04 proper? |
|
Thanks @gary88christie. It appears to be ok as far as I can tell. Can you submit a pull request and I'll try it out? |
Track 1 & Track 2 can run successfully.
But I met a Track 3 docker evironment error because nvidia/cuda:11.0-devel-ubuntu18.04 is no longer available.
So I tried with a similar nvidia docker version nvidia/cuda:11.0.3-devel-ubuntu18.04, but seems to cause a lot of package conflicts:
The text was updated successfully, but these errors were encountered: