Skip to content

Conversation

@MisterDA
Copy link
Contributor

Try to reorder calls to update the package lists, upgrade the current installed packages, and install new packages to minimize network calls. Some fixes along the way. Make sure that installing packages updates the package lists first.
Most package managers don't have a package download cache enabled, but pacman does. Cleaning the pacman dl cache recovers ~112M (with libx11 as a dependency, YMMV: depends whether new packages have been published since the build of the base image).
Some package managers require package lists to be available before upgrading/installing packages (apt), others will automatically download them. Be conservative, and keep the package lists in all cases.
opam can't download the package lists if the package manager doesn't do it automatically, and will fail the opam depext install if so.
There's no particular need to crunch the lines into simpler layers, since the images are later squashed.

@MisterDA MisterDA force-pushed the optimize-pkg-install branch from 64b9d46 to 824a75a Compare March 13, 2023 13:01
@MisterDA MisterDA merged commit f9bd142 into master Mar 13, 2023
@MisterDA MisterDA deleted the optimize-pkg-install branch March 13, 2023 13:01
@kit-ty-kate
Copy link
Contributor

I’m not sure why I was asked for a review if it’s going to be merged the next day (weekend) without any input.

There is a problem on distributions using yum: yum clean all removes the package database.

@MisterDA
Copy link
Contributor Author

I’m not sure why I was asked for a review if it’s going to be merged the next day (weekend) without any input.

You're right, I should have left this waiting longer, sorry.

There is a problem on distributions using yum: yum clean all removes the package database.

(This wasn't changed in this PR)
That's right, but yum and dnf have the courtesy of fetching the package database if it doesn't exist, so opam depext will not fail. yum clean packages is probably more efficient. The package database for Fedora is ~110Mb, so there's a case for including it in the image even if it's heavy, to avoid re-downloading it every time.

@kit-ty-kate
Copy link
Contributor

(This wasn't changed in this PR)

it was because the database was redownloaded at the next yum install just after.

@MisterDA MisterDA mentioned this pull request Mar 14, 2023
MisterDA added a commit to MisterDA/opam-repository that referenced this pull request Mar 23, 2023
CHANGES:

- Install system packages required by OCaml in the ocaml stage,
  starting with OCaml 5.1 and libzstd.
  (@MisterDA ocurrent/ocaml-dockerfile#149, review by @kit-ty-kate)
- Add OracleLinux 9. (@MisterDA ocurrent/ocaml-dockerfile#155)
- Optimize and fix Linux package install.
  (@MisterDA ocurrent/ocaml-dockerfile#147, ocurrent/ocaml-dockerfile#151, ocurrent/ocaml-dockerfile#153, ocurrent/ocaml-dockerfile#154, review by @kit-ty-kate)
- Switch to ocaml-opam/opam-repository-mingw#sunset for Windows images. (@MisterDA ocurrent/ocaml-dockerfile#152)
- Use DockerHub user risvc64/ubuntu. (@MisterDA, ocurrent/ocaml-dockerfile#150)
- Various LCU Updates (@mtelvers ocurrent/ocaml-dockerfile#144 ocurrent/ocaml-dockerfile#136 ocurrent/ocaml-dockerfile#135)
- Support mounts, networks, and security parameters in RUN
  commands, add buildkit_syntax helper function.
  (@MisterDA, @edwintorok, ocurrent/ocaml-dockerfile#137, ocurrent/ocaml-dockerfile#139, review by @edwintorok)
- Build and install opam master from source in Windows images.
  (@MisterDA ocurrent/ocaml-dockerfile#140, ocurrent/ocaml-dockerfile#142, ocurrent/ocaml-dockerfile#143)
- Include the ocaml-beta-repository in the images. (@kit-ty-kate ocurrent/ocaml-dockerfile#132, review by @MisterDA)
- Add OpenSUSE 15.4, deprecate OpenSUSE 15.3. (@MisterDA ocurrent/ocaml-dockerfile#138)
- Update to bubblewrap 0.8.0. (@MisterDA ocurrent/ocaml-dockerfile#131 ocurrent/ocaml-dockerfile#148)
- Add Alpine 3.17 (3.16 is now tier 2 and 3.15 is deprecated). Remove
  libexecinfo-dev from the list of apk packages as it is no longer
  available. Its symbols are only used in OCaml's self tests.
  (@MisterDA ocurrent/ocaml-dockerfile#129, ocurrent/ocaml-dockerfile#130)
- Fix location of Debian exotic architecture images (@dra27 ocurrent/ocaml-dockerfile#134)
- Fix passing of --platform to all stages of the Dockerfiles (@dra27 ocurrent/ocaml-dockerfile#134)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants