Build more OTP versions?

[josevalim]

Hi @c0b!

Thank you for all for the work on maintaining the Docker images!

Now that GitHub Actions is using containers, I expect developers to want to use more combinations of Elixir+OTP. For deployment, it is fine (and even recommended) to push to the latest versions, but as a maintainer of Ecto/Phoenix/etc, I may want to test more combinations (for example, 1.9+22.3 and 1.9+22.0). In this case, would it be possible for you build images for each support OTP branch? For example, for v1.9, it would be 1.9+20.3, 1.9+21.3, and 1.9+22-latest. The complete list can always be found here: https://hexdocs.pm/elixir/compatibility-and-deprecations.html#compatibility-between-elixir-and-erlang-otp

Btw, I have no idea how hard this actually is, so if for some reason this is tons of work, then feel free to close this.

Thanks!

[tsloughter]

@josevalim weren't you the one who suggested the model of downloading pre-built Elixirs from github instead of compiling on different OTP versions in the first place ;)

I think these should definitely change to compile on specific otps.

And if @elixir-lang doesn't want to take ownership maybe we could merge this into https://github.com/erlang/docker-erlang-otp

Then under each dir like 22/ have a build for elixir on that OTP version.

[josevalim]

@tsloughter yeah and we can still use the prebuilt ones by Hex, which are compiled for each OTP version.

And I like that this is not centralized under the Elixir team, plus @c0b is doing a fantastic job. Btw, you want to update your link above. :D

[tsloughter]

Hah, goddammit, one second.

[tsloughter]

Oh, I thought the issue people had with these images was that they weren't built with the latest OTP version because they were the pre-built tarballs that are only provided for running the earliest supported OTP or whatever.

[josevalim]

The one provided by Elixir in GitHub yes, but Hex also prebuilds them: https://github.com/hexpm/bob/blob/master/README.md#elixir-builds

[wojtekmach]

For deployment, it is fine (and even recommended) to push to the latest versions, but as a maintainer of Ecto/Phoenix/etc, I may want to test more combinations (for example, 1.9+22.3 and 1.9+22.0).

hexpm/bob already built the following combinations for Elixir v1.9:

curl --silent https://repo.hex.pm/builds/elixir/builds.txt | grep v1.9 | cut -d" " -f1
v1.9
v1.9-otp-20
v1.9-otp-21
v1.9-otp-22
v1.9.0
v1.9.0-otp-20
v1.9.0-otp-21
v1.9.0-otp-22
v1.9.0-rc.0
v1.9.0-rc.0-otp-20
v1.9.0-rc.0-otp-21
v1.9.0-rc.0-otp-22
v1.9.1
v1.9.1-otp-20
v1.9.1-otp-21
v1.9.1-otp-22

if we are to build docker images against minor OTP versions and variants (alpine and slim), well, that's a huge number of images. Maybe that's ok, or maybe we should pick specific combinations? If so, what heuristic should we use?

Btw, I put together a project [1] that generates and publishes [2] docker images for both Erlang and Elixir. One kinda nice thing is you can specify desired combinations [3] and a script generates appropriate dockerfiles, builds them, and pushes them. Maybe such script could be useful for official images for either language.

[1] https://github.com/wojtekmach/docker-beam
[2] https://hub.docker.com/u/wojtekmach
[3] https://github.com/wojtekmach/docker-beam/blob/master/builds.txt

[jwietelmann]

At the very least, it's important to start pinning the minor version of the erlang image. Everyone just got an accidental update from OTP 22.0 to OTP 22.1 which broke hackney: benoitc/hackney#591

[vaxinate]

@jwietelmann beat me to the punch. It's also worth noting that we've been bitten by this more than once across different projects w/in our company. We don't necessarily need every permutation of elixir/OTP releases, but we need to be able to lock ourselves into one w/o having to worry about whether we're going to get surprised by an OTP version change.

[c0b]

Then under each dir like 22/ have a build for elixir on that OTP version.

then what's further directory layout under 22/? is it a separate repo erlang/docker-erlang-otp#245 (comment)

22.0-elixir-alpine
22.0-elixir
22.1-elixir-alpine
22.1-elixir
...

At the very least, it's important to start pinning the minor version of the erlang image. Everyone just got an accidental update from OTP 22.0 to OTP 22.1 which broke hackney: benoitc/hackney#591

yes, probably not every permutation but when some of the breaking minor version changes, it would deserve branch out a variant permutation

[maennchen]

I spent some time to explore using bob to build all images. Both mentioned PR's are WIP. (README, stackbrew, Travis CI are untouched...)

• Use Bob to build the existing images (Use Hex.pm/Bob to build images #126)
• Use Bob Manifest to build all elixir / erlang permutations existing in the build.txt (Use Hex.pm/Bob to build all available versions #127)

I'd love to get some feedback on that.

[maennchen]

@c0b Did you have time to take a look?

[tsloughter]

Shouldn't need this now that https://hub.docker.com/r/hexpm/elixir exists.

I think we should see about getting hexpm/elixir to be the official images. Just not sure if the "docker official" image rules for naming will be an issue?

@c0b what do you think?

Only issue I see right now, except for possibly Docker requiring different image tags, is they only have alpine right now.

[bettio]

Hello @c0b, do you have any plan for OTP-23 images?

I would love to start using as soon as possible OTP 23 with Elixir 1.10.x to validate it, so we can start using it in production soon (it looks like OTP 23 has great performance improvements on kubernetes...).
However I think it would be unfair for Elixir 1.10.x image users a switch from OTP22 to OTP23 without any notice on a minor version update, on the other hand Elixir 1.11 will be out on next fall so it would be a long wait.
Can we have any OTP23 flavor Elixir 1.10.x image?

[conradwt]

@bettio If you need it now, I have created the following Alpine Dockerfile from this repository:

Sample Dockerfile

FROM erlang:23.0.2-alpine

# elixir expects utf8.
ENV ELIXIR_VERSION="v1.10.3" \
  LANG=C.UTF-8

RUN set -xe \
  && ELIXIR_DOWNLOAD_URL="https://github.com/elixir-lang/elixir/archive/${ELIXIR_VERSION}.tar.gz" \
  && ELIXIR_DOWNLOAD_SHA256="f3035fc5fdade35c3592a5fa7c8ee1aadb736f565c46b74b68ed7828b3ee1897" \
  && buildDeps=' \
  ca-certificates \
  curl \
  make \
  ' \
  && apk add --no-cache --virtual .build-deps $buildDeps \
  && curl -fSL -o elixir-src.tar.gz $ELIXIR_DOWNLOAD_URL \
  && echo "$ELIXIR_DOWNLOAD_SHA256  elixir-src.tar.gz" | sha256sum -c - \
  && mkdir -p /usr/local/src/elixir \
  && tar -xzC /usr/local/src/elixir --strip-components=1 -f elixir-src.tar.gz \
  && rm elixir-src.tar.gz \
  && cd /usr/local/src/elixir \
  && make install clean \
  && apk del .build-deps

CMD ["iex"]

Build and Push

docker build -t <username>/elixir:1.10.3-alpine .
docker push <username>/elixir:1.10.3-alpine

Final Thoughts

In the above, I used Alpine but please feel free to change the operating system (OS) based on your project requirements.