Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(build): Update dockerfile to use cache busting and reduce image size #8652

Merged
merged 4 commits into from
Feb 7, 2023
Merged

fix(build): Update dockerfile to use cache busting and reduce image size #8652

merged 4 commits into from
Feb 7, 2023

Conversation

all-seeing-code
Copy link
Contributor

@all-seeing-code all-seeing-code commented Feb 6, 2023 •
edited
Loading

This PR fixes issue in building docker-images during CI and releases. It uses cache busting to ensure apt-get update is always run. It also reduces the image size by removing /var/lib/apt/lists

See: RUN in https://docs.docker.com/develop/develop-images/dockerfile_best-practices/

@github-actions github-actions bot added the area/integrations Related to integrations with other projects. label Feb 6, 2023
@all-seeing-code all-seeing-code changed the title Update dockerfile as per best practices fix(build): Update dockerfile to use cache busting and reduce image size Feb 6, 2023
skrdgraph
skrdgraph approved these changes Feb 6, 2023
View reviewed changes
Copy link
Contributor

@skrdgraph skrdgraph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks okay to merge. Approving - (BUT please wait for the ubuntu repo issues to resolve before merging).

Some things to keep in mind (if we see issues):

  • this would create one big layer
  • separate layers is less load on the infrastructure
  • so bundling into 1 layer could cause network contention
  • also separate layers could help in faster image pulling, as other containers may share the same base layers.

@skrdgraph skrdgraph self-requested a review February 6, 2023 17:06
@mangalaman93
Copy link
Member

@anurags92 could you check the size of this layer before merging this PR? It should be a reasonable size, probably <50MB

@coveralls
Copy link

Coverage Status

Coverage: 67.219% (+0.5%) from 66.74% when pulling 03ce70e on anurag/fix-docker-image-build into 43f1d8a on main.

@all-seeing-code
Copy link
Contributor Author

This looks okay to merge. Approving - (BUT please wait for the ubuntu repo issues to resolve before merging).

Some things to keep in mind (if we see issues):

  • this would create one big layer
  • separate layers is less load on the infrastructure
  • so bundling into 1 layer could cause network contention
  • also separate layers could help in faster image pulling, as other containers may share the same base layers.

Good points, I checked the individual image sizes and layers.

  • With changes in current PR:
| REPOSITORY    | TAG   | IMAGE ID     | CREATED | SIZE  |
|---------------|-------|--------------|---------|-------|
| dgraph/dgraph | local | 5036cedd39aa |         | 156MB |

| IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT |
|---------------------------------------------------------------------------------------------------|
| c51beb60c424   56 seconds ago   /bin/sh -c #(nop)  CMD ["dgraph"]               0B                |
| df17c82206cb   56 seconds ago   /bin/sh -c #(nop)  ENV GODEBUG=madvdontneed=1   0B                |
| 43fa4acdb279   57 seconds ago   /bin/sh -c #(nop) WORKDIR /dgraph               0B                |
| da5148b1c232   57 seconds ago   /bin/sh -c mkdir /dgraph                        0B                |
| 186f5ef257dd   58 seconds ago   /bin/sh -c #(nop)  EXPOSE 9080                  0B                |
| b4cce2762407   58 seconds ago   /bin/sh -c #(nop)  EXPOSE 8080                  0B                |
| 3f52a101f5bf   58 seconds ago   /bin/sh -c #(nop) ADD dir:bde9606849994d9561…   64.7MB            |
| d19f57b2e085   59 seconds ago   /bin/sh -c apt-get update && apt-get install…   18.6MB            |
| 91b1753a0b20   5 weeks ago      /bin/sh -c rm -rf /var/lib/apt/lists/*          0B                |
| b7d521842518   6 weeks ago      /bin/sh -c #(nop)  LABEL maintainer=Dgraph L…   0B                |
| 817578334b4d   4 months ago     /bin/sh -c #(nop)  CMD ["bash"]                 0B                |
| <missing>      4 months ago     /bin/sh -c #(nop) ADD file:8faed18d471598732…   72.8MB            |
  • With current docker file in main
| REPOSITORY    | TAG   | IMAGE ID                 | CREATED | SIZE  |
|---------------|-------|--------------------------|---------|-------|
| dgraph/dgraph | local | 5036cedd1a228902b5ad39aa |         | 203MB |

| IMAGE          CREATED              CREATED BY                                      SIZE      COMMENT |
|-------------------------------------------------------------------------------------------------------|
| d1a228902b5a   About a minute ago   /bin/sh -c #(nop)  CMD ["dgraph"]               0B                |
| b6aeb8eb188a   About a minute ago   /bin/sh -c #(nop)  ENV GODEBUG=madvdontneed=1   0B                |
| 453def3fa27e   About a minute ago   /bin/sh -c #(nop) WORKDIR /dgraph               0B                |
| beb746ba967e   About a minute ago   /bin/sh -c mkdir /dgraph                        0B                |
| 53c0f2a54ad6   About a minute ago   /bin/sh -c #(nop)  EXPOSE 9080                  0B                |
| 6e1832cf29b4   About a minute ago   /bin/sh -c #(nop)  EXPOSE 8080                  0B                |
| 0cd0e85af060   About a minute ago   /bin/sh -c #(nop) ADD dir:a133d7a1a314059ec4…   64.7MB            |
| 0093b23b9588   About a minute ago   /bin/sh -c apt-get install -y --no-install-r…   2.91MB            |
| 19a09a866dae   2 minutes ago        /bin/sh -c apt-get install -y --no-install-r…   1.24MB            |
| c585f0d8c868   2 minutes ago        /bin/sh -c apt-get install -y --no-install-r…   2MB               |
| 47875777746c   2 minutes ago        /bin/sh -c apt-get install -y --no-install-r…   1.21MB            |
| d7ddbb44a190   2 minutes ago        /bin/sh -c apt-get install -y --no-install-r…   1.18MB            |
| 4f710b0f70e3   2 minutes ago        /bin/sh -c apt-get install -y --no-install-r…   9.94MB            |
| e0da33a79b9f   2 minutes ago        /bin/sh -c apt-get install -y --no-install-r…   6.59MB            |
| 599a6292153d   2 minutes ago        /bin/sh -c apt-get update --fix-missing         40.5MB            |
| 91b1753a0b20   5 weeks ago          /bin/sh -c rm -rf /var/lib/apt/lists/*          0B                |
| b7d521842518   6 weeks ago          /bin/sh -c #(nop)  LABEL maintainer=Dgraph L…   0B                |
| 817578334b4d   4 months ago         /bin/sh -c #(nop)  CMD ["bash"]                 0B                |
| <missing>      4 months ago         /bin/sh -c #(nop) ADD file:8faed18d471598732…   72.8MB            |

@all-seeing-code
Copy link
Contributor Author

all-seeing-code commented Feb 7, 2023 •
edited
Loading

This confirms that layering works based on previous SHAs etc.

This looks okay to merge. Approving - (BUT please wait for the ubuntu repo issues to resolve before merging).
Some things to keep in mind (if we see issues):

  • this would create one big layer
  • separate layers is less load on the infrastructure
  • so bundling into 1 layer could cause network contention
  • also separate layers could help in faster image pulling, as other containers may share the same base layers.

Good points, I checked the individual image sizes and layers.

  • With changes in current PR:
| REPOSITORY    | TAG   | IMAGE ID     | CREATED | SIZE  |
|---------------|-------|--------------|---------|-------|
| dgraph/dgraph | local | 5036cedd39aa |         | 156MB |

| IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT |
|---------------------------------------------------------------------------------------------------|
| c51beb60c424   56 seconds ago   /bin/sh -c #(nop)  CMD ["dgraph"]               0B                |
| df17c82206cb   56 seconds ago   /bin/sh -c #(nop)  ENV GODEBUG=madvdontneed=1   0B                |
| 43fa4acdb279   57 seconds ago   /bin/sh -c #(nop) WORKDIR /dgraph               0B                |
| da5148b1c232   57 seconds ago   /bin/sh -c mkdir /dgraph                        0B                |
| 186f5ef257dd   58 seconds ago   /bin/sh -c #(nop)  EXPOSE 9080                  0B                |
| b4cce2762407   58 seconds ago   /bin/sh -c #(nop)  EXPOSE 8080                  0B                |
| 3f52a101f5bf   58 seconds ago   /bin/sh -c #(nop) ADD dir:bde9606849994d9561…   64.7MB            |
| d19f57b2e085   59 seconds ago   /bin/sh -c apt-get update && apt-get install…   18.6MB            |
| 91b1753a0b20   5 weeks ago      /bin/sh -c rm -rf /var/lib/apt/lists/*          0B                |
| b7d521842518   6 weeks ago      /bin/sh -c #(nop)  LABEL maintainer=Dgraph L…   0B                |
| 817578334b4d   4 months ago     /bin/sh -c #(nop)  CMD ["bash"]                 0B                |
| <missing>      4 months ago     /bin/sh -c #(nop) ADD file:8faed18d471598732…   72.8MB            |
  • With current docker file in main
| REPOSITORY    | TAG   | IMAGE ID                 | CREATED | SIZE  |
|---------------|-------|--------------------------|---------|-------|
| dgraph/dgraph | local | 5036cedd1a228902b5ad39aa |         | 203MB |

| IMAGE          CREATED              CREATED BY                                      SIZE      COMMENT |
|-------------------------------------------------------------------------------------------------------|
| d1a228902b5a   About a minute ago   /bin/sh -c #(nop)  CMD ["dgraph"]               0B                |
| b6aeb8eb188a   About a minute ago   /bin/sh -c #(nop)  ENV GODEBUG=madvdontneed=1   0B                |
| 453def3fa27e   About a minute ago   /bin/sh -c #(nop) WORKDIR /dgraph               0B                |
| beb746ba967e   About a minute ago   /bin/sh -c mkdir /dgraph                        0B                |
| 53c0f2a54ad6   About a minute ago   /bin/sh -c #(nop)  EXPOSE 9080                  0B                |
| 6e1832cf29b4   About a minute ago   /bin/sh -c #(nop)  EXPOSE 8080                  0B                |
| 0cd0e85af060   About a minute ago   /bin/sh -c #(nop) ADD dir:a133d7a1a314059ec4…   64.7MB            |
| 0093b23b9588   About a minute ago   /bin/sh -c apt-get install -y --no-install-r…   2.91MB            |
| 19a09a866dae   2 minutes ago        /bin/sh -c apt-get install -y --no-install-r…   1.24MB            |
| c585f0d8c868   2 minutes ago        /bin/sh -c apt-get install -y --no-install-r…   2MB               |
| 47875777746c   2 minutes ago        /bin/sh -c apt-get install -y --no-install-r…   1.21MB            |
| d7ddbb44a190   2 minutes ago        /bin/sh -c apt-get install -y --no-install-r…   1.18MB            |
| 4f710b0f70e3   2 minutes ago        /bin/sh -c apt-get install -y --no-install-r…   9.94MB            |
| e0da33a79b9f   2 minutes ago        /bin/sh -c apt-get install -y --no-install-r…   6.59MB            |
| 599a6292153d   2 minutes ago        /bin/sh -c apt-get update --fix-missing         40.5MB            |
| 91b1753a0b20   5 weeks ago          /bin/sh -c rm -rf /var/lib/apt/lists/*          0B                |
| b7d521842518   6 weeks ago          /bin/sh -c #(nop)  LABEL maintainer=Dgraph L…   0B                |
| 817578334b4d   4 months ago         /bin/sh -c #(nop)  CMD ["bash"]                 0B                |
| <missing>      4 months ago         /bin/sh -c #(nop) ADD file:8faed18d471598732…   72.8MB            |

| IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT |
|---------------------------------------------------------------------------------------------------|
| 16ff6fa05bc1   8 seconds ago    /bin/sh -c #(nop)  CMD ["dgraph"]               0B                |
| 2993b409a372   9 seconds ago    /bin/sh -c #(nop)  ENV GODEBUG=madvdontneed=1   0B                |
| 150b9fc0385c   9 seconds ago    /bin/sh -c #(nop) WORKDIR /dgraph               0B                |
| 367c460ea8ff   9 seconds ago    /bin/sh -c mkdir /dgraph                        0B                |
| 6f0f6fd125f7   10 seconds ago   /bin/sh -c #(nop)  EXPOSE 9080                  0B                |
| a11d7fe9a19e   10 seconds ago   /bin/sh -c #(nop)  EXPOSE 8080                  0B                |
| 07e03e4d46b6   10 seconds ago   /bin/sh -c #(nop) ADD dir:a133d7a1a314059ec4…   64.7MB            |
| bab9269f6d8b   11 seconds ago   /bin/sh -c apt-get install -y --no-install-r…   2.91MB            |
| ee0dff5fd0f2   17 seconds ago   /bin/sh -c apt-get install -y --no-install-r…   1.24MB            |
| 472f3e0c943c   20 seconds ago   /bin/sh -c apt-get install -y --no-install-r…   2MB               |
| 75e1a6129ba9   24 seconds ago   /bin/sh -c apt-get install -y --no-install-r…   1.21MB            |
| be63457ce15e   29 seconds ago   /bin/sh -c apt-get install -y htop              1.18MB            |
| 4f710b0f70e3   26 minutes ago   /bin/sh -c apt-get install -y --no-install-r…   9.94MB            |
| e0da33a79b9f   26 minutes ago   /bin/sh -c apt-get install -y --no-install-r…   6.59MB            |
| 599a6292153d   26 minutes ago   /bin/sh -c apt-get update --fix-missing         40.5MB            |
| 91b1753a0b20   5 weeks ago      /bin/sh -c rm -rf /var/lib/apt/lists/*          0B                |
| b7d521842518   6 weeks ago      /bin/sh -c #(nop)  LABEL maintainer=Dgraph L…   0B                |
| 817578334b4d   4 months ago     /bin/sh -c #(nop)  CMD ["bash"]                 0B                |
| <missing>      4 months ago     /bin/sh -c #(nop) ADD file:8faed18d471598732…   72.8MB            |

@all-seeing-code all-seeing-code merged commit 695354d into main Feb 7, 2023
@all-seeing-code all-seeing-code deleted the anurag/fix-docker-image-build branch February 7, 2023 08:41
all-seeing-code added a commit that referenced this pull request Feb 8, 2023
@all-seeing-code
fix(build): Update dockerfile to use cache busting and reduce image s…
6499c3f 
…ize (#8652)

This PR fixes issue in building docker-images during CI and releases. It
uses cache busting to ensure `apt-get update` is always run. It also
reduces the image size by removing `/var/lib/apt/lists`
all-seeing-code added a commit that referenced this pull request Feb 8, 2023
@all-seeing-code
fix(build): Update dockerfile to use cache busting and reduce image s…
ba06421 
…ize (#8652)

This PR fixes issue in building docker-images during CI and releases. It
uses cache busting to ensure `apt-get update` is always run. It also
reduces the image size by removing `/var/lib/apt/lists`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@skrdgraph skrdgraph skrdgraph approved these changes

@mangalaman93 mangalaman93 mangalaman93 approved these changes

@darkn3rd darkn3rd Awaiting requested review from darkn3rd

Assignees
No one assigned
Labels
area/integrations Related to integrations with other projects.
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@all-seeing-code @mangalaman93 @coveralls @skrdgraph