Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

CI: Consolidate Dockerfiles #18115

Merged
merged 1 commit into from
Apr 22, 2020
Merged

CI: Consolidate Dockerfiles #18115

merged 1 commit into from
Apr 22, 2020

Conversation

leezu
Copy link
Contributor

@leezu leezu commented Apr 20, 2020

Description

This PR shows how to get rid of duplicate Dockerfiles and the install/X.sh code deduplication strategy on the example of CentOS7 Dockerfiles. Instead, we use a single templated Dockerfile with a multiple targets for different scenarios. The template arguments and respective targets are declared in docker-compose.yml.

This has multiple benefits:

  1. Having a single Dockerfile instead of a number of scripts is easier to understand and maintain
  2. Copying scripts into the Dockerfile as done in the previous approach prevents usage of the https://hub.docker.com/u/mxnetci/ cache on developer machines, as the COPY instruction invalidates the cache if the local repository does not match the repository used on CI (eg. development branch instead of master branch). If the cache is invalidated, developers must wait for long docker build time when attempting to reproduce certain CI builds (such as website build, cc @aaronmarkham) or using the docker image to perform the staticbuild.
  3. Having fewer steps in the Dockerfile improves the build speed.

@mxnet-bot
Copy link

Hey @leezu , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [windows-gpu, windows-cpu, miscellaneous, website, centos-gpu, unix-gpu, edge, sanity, centos-cpu, clang, unix-cpu]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@marcoabreu
Copy link
Contributor

That looks really neat!

@leezu

This comment has been minimized.

@mxnet-bot

This comment has been minimized.

@leezu leezu force-pushed the refactordocker branch 6 times, most recently from e75c213 to d3ef052 Compare April 21, 2020 04:55
@leezu leezu added the pr-awaiting-review PR is waiting for code review label Apr 21, 2020
@szha szha merged commit b85e446 into apache:master Apr 22, 2020
@leezu leezu deleted the refactordocker branch April 22, 2020 00:35
AntiZpvoh pushed a commit to AntiZpvoh/incubator-mxnet that referenced this pull request Jul 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
pr-awaiting-review PR is waiting for code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants