Fix apt repo issue for docker #3823

ydcjeff · 2020-10-03T17:32:42Z

What does this PR do?

Fixes # (issue)

Before submitting

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?
Did you verify new and existing tests pass locally with your changes?
If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

ydcjeff · 2020-10-03T18:12:11Z

Docker builds relating with nightly will fail at the moment since pytorch has released 1.7 rc version.

Borda · 2020-10-03T18:28:36Z

dockers/base-cuda/Dockerfile

@@ -34,7 +34,7 @@ SHELL ["/bin/bash", "-c"]

 ENV PATH="$PATH:/root/.local/bin"

-RUN apt-get update && apt-get install -y --no-install-recommends \
+RUN apt-get clean && apt-get update && apt-get install -y --no-install-recommends \


This seems strange, you need update before install not clean..

Suggested change

RUN apt-get clean && apt-get update && apt-get install -y --no-install-recommends \

RUN apt-get update && apt-get install -y --no-install-recommends \

Turns out it's CDN issue, some suggest clean and update
but let's wait and see if CDN update can solve this

NVIDIA/nvidia-docker#1392
NVIDIA/nvidia-docker#877
NVIDIA/nvidia-docker#1328

ok, the problem is in

Reading package lists... E: Failed to fetch https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/Packages.gz File has unexpected size (47871 != 49498). Mirror sync in progress? [IP: 152.195.19.142 443] Hashes of expected file: - Filesize:49498 [weak] - SHA256:332f3ee4e353b8a5e5a2bdd8fdbd47cf140c73822b82b328815f122e09e195a0 - SHA1:4dc8ef9a3ee3c97b3c26d46e07fdd83997e6880b [weak] - MD5Sum:bbff3b9c3462257479d72521ee78ec29 [weak] Release file created at: Wed, 23 Sep 2020 22:09:13 +0000 E: Some index files failed to download. They have been ignored, or old ones used instead.

not the update step...

That comes from this, I think

Get:10 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 Packages [49.5 kB] Err:10 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 Packages File has unexpected size (47871 != 49498). Mirror sync in progress? [IP: 152.195.19.142 443] Hashes of expected file: - Filesize:49498 [weak] - SHA256:332f3ee4e353b8a5e5a2bdd8fdbd47cf140c73822b82b328815f122e09e195a0 - SHA1:4dc8ef9a3ee3c97b3c26d46e07fdd83997e6880b [weak] - MD5Sum:bbff3b9c3462257479d72521ee78ec29 [weak] Release file created at: Wed, 23 Sep 2020 22:09:13 +0000

so skip of the update is just hiding the NVIDIA package issue not solving it.. :[

Borda · 2020-10-04T08:06:48Z

dockers/base-xla/Dockerfile

@@ -28,7 +28,7 @@ ENV CONDA_ENV=lightning
 # show system inforation
 RUN lsb_release -a && cat /etc/*-release

-RUN apt-get update && \
+RUN apt-get clean && apt-get update && \


Suggested change

RUN apt-get clean && apt-get update && \

RUN apt-get update && \

Borda · 2020-10-04T08:07:59Z

dockers/base-cuda/Dockerfile

@@ -34,7 +34,7 @@ SHELL ["/bin/bash", "-c"]

 ENV PATH="$PATH:/root/.local/bin"

-RUN apt-get update && apt-get install -y --no-install-recommends \
+RUN apt-get clean && apt-get update && apt-get install -y --no-install-recommends \


ok, the problem is in

Reading package lists... E: Failed to fetch https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/Packages.gz File has unexpected size (47871 != 49498). Mirror sync in progress? [IP: 152.195.19.142 443] Hashes of expected file: - Filesize:49498 [weak] - SHA256:332f3ee4e353b8a5e5a2bdd8fdbd47cf140c73822b82b328815f122e09e195a0 - SHA1:4dc8ef9a3ee3c97b3c26d46e07fdd83997e6880b [weak] - MD5Sum:bbff3b9c3462257479d72521ee78ec29 [weak] Release file created at: Wed, 23 Sep 2020 22:09:13 +0000 E: Some index files failed to download. They have been ignored, or old ones used instead.

not the update step...

codecov · 2020-10-04T08:34:08Z

Codecov Report

Merging #3823 into master will decrease coverage by 4%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #3823    +/-   ##
=======================================
- Coverage      84%     80%    -4%     
=======================================
  Files         111     111            
  Lines        8793    9379   +586     
=======================================
+ Hits         7362    7515   +153     
- Misses       1431    1864   +433

ydcjeff · 2020-10-04T09:34:09Z

Is it okay to use CUDA 11?

ydcjeff · 2020-10-05T12:17:40Z

Downgrading to ubuntu 16.04 works fine

Borda

pls, no 5 years old OS

Borda · 2020-10-06T08:50:10Z

dockers/base-cuda/Dockerfile

-# FROM nvidia/cuda:${CUDA_VERSION}-devel
+# FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu20.04
+# FROM nvidia/cuda:${CUDA_VERSION}-cudnn${CUDNN_VERSION}-devel-ubuntu18.04
+FROM nvidia/cuda:${CUDA_VERSION}-cudnn${CUDNN_VERSION}-devel-ubuntu16.04


this is bad, this is too. old OS version, this is the almost unsupported OS (ends in April 2021)

fix docker repo issue

9813eb3

mergify bot requested a review from a team October 3, 2020 17:33

Borda reviewed Oct 3, 2020

View reviewed changes

mergify bot requested a review from a team October 3, 2020 18:28

Borda reviewed Oct 4, 2020

View reviewed changes

mergify bot requested a review from a team October 4, 2020 08:07

Borda requested changes Oct 4, 2020

View reviewed changes

mergify bot requested a review from a team October 4, 2020 08:08

docker

b411f06

Borda added 3rd party Related to a 3rd-party ci Continuous Integration labels Oct 4, 2020

Borda added 2 commits October 4, 2020 10:38

docker

80be0e0

docker

3e70d24

Borda added the priority: 0 High priority task label Oct 4, 2020

Borda approved these changes Oct 4, 2020

View reviewed changes

mergify bot requested a review from a team October 4, 2020 08:57

justusschock approved these changes Oct 5, 2020

View reviewed changes

mergify bot requested a review from a team October 5, 2020 07:50

ydcjeff added 3 commits October 5, 2020 18:04

no cudnn

c14f360

no cudnn

192c13c

try 16.04

63c7617

SkafteNicki approved these changes Oct 5, 2020

View reviewed changes

Borda requested changes Oct 5, 2020

View reviewed changes

mergify bot requested a review from a team October 5, 2020 14:11

williamFalcon merged commit 90929fa into Lightning-AI:master Oct 6, 2020

ydcjeff deleted the docker/fix-nvidia branch October 6, 2020 04:08

Borda reviewed Oct 6, 2020

View reviewed changes

Borda removed the priority: 0 High priority task label Oct 6, 2020

mergify bot requested a review from a team October 6, 2020 08:50

Borda mentioned this pull request Oct 6, 2020

use fine Ubuntu #3896

Merged

Borda added this to the 0.10.0 milestone Oct 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix apt repo issue for docker #3823

Fix apt repo issue for docker #3823

ydcjeff commented Oct 3, 2020

ydcjeff commented Oct 3, 2020

Borda Oct 3, 2020 •

edited

Loading

ydcjeff Oct 4, 2020 •

edited

Loading

Borda Oct 4, 2020

ydcjeff Oct 4, 2020

Borda Oct 4, 2020

Borda Oct 4, 2020

Borda Oct 4, 2020

codecov bot commented Oct 4, 2020 •

edited

Loading

ydcjeff commented Oct 4, 2020

ydcjeff commented Oct 5, 2020

Borda left a comment

Borda Oct 6, 2020

	RUN apt-get clean && apt-get update && apt-get install -y --no-install-recommends \
	RUN apt-get update && apt-get install -y --no-install-recommends \

	RUN apt-get clean && apt-get update && \
	RUN apt-get update && \

Fix apt repo issue for docker #3823

Fix apt repo issue for docker #3823

Conversation

ydcjeff commented Oct 3, 2020

What does this PR do?

Before submitting

PR review

Did you have fun?

ydcjeff commented Oct 3, 2020

Borda Oct 3, 2020 • edited Loading

Choose a reason for hiding this comment

ydcjeff Oct 4, 2020 • edited Loading

Choose a reason for hiding this comment

Borda Oct 4, 2020

Choose a reason for hiding this comment

ydcjeff Oct 4, 2020

Choose a reason for hiding this comment

Borda Oct 4, 2020

Choose a reason for hiding this comment

Borda Oct 4, 2020

Choose a reason for hiding this comment

Borda Oct 4, 2020

Choose a reason for hiding this comment

codecov bot commented Oct 4, 2020 • edited Loading

Codecov Report

ydcjeff commented Oct 4, 2020

ydcjeff commented Oct 5, 2020

Borda left a comment

Choose a reason for hiding this comment

Borda Oct 6, 2020

Choose a reason for hiding this comment

Borda Oct 3, 2020 •

edited

Loading

ydcjeff Oct 4, 2020 •

edited

Loading

codecov bot commented Oct 4, 2020 •

edited

Loading