Explore ways to trim down the image size

[pesho]

Some ideas suggested elsewhere:

1. Use a smaller base image for the default variant, e.g. buildpack-deps:jessie-micro
2. The build-essential package might be too fat, as it also brings stuff for Debian package development which we don't need. We may save space by installing only the useful bits.
3. The packages needed for building native modules might be left out (with a warning to the users that they should install them themselves if required).

[pesho]

For the record, I'm -1 on point 3, +1 on the rest (if testing shows real savings).

[tianon]

FWIW, I'm only +1 on point 3 for the slim variant -- in the default image I definitely agree that they ought to be there, and I'd actually highly recommend you stick to the fat buildpack-deps there too (since it specifically includes the C headers necessary for the vast majority of native modules to be compiled nicely)

[retrohacker]

I am 👍 atm for 3. Use case:

cd ${APP_DIR}
npm install
docker run -v ${PWD}:/usr/src/app -w /usr/src/app --it --rm iojs npm start

You don't always want to run npm install inside the container. A lot of applications fail quick and restart, this works well with that.

[pesho]

My main objection is that this would change the provided functionality of the -slim image in a big way. It's currently a drop-in replacement for the default variant, in the vast majority of usecases, and people may already rely on that (from the node image).

Can we instead introduce a new variant, e.g. -tiny?

[tianon]

Please don't introduce a new variant. It's not supposed to be a 100% drop-in replacement for the default variant. What we're discussing is what the "slim" variant was supposed to be from the beginning.

[pesho]

What we're discussing is what the "slim" variant was supposed to be from the beginning.

In that case I change my -1 to 0 😐

[tianon]

It could be that some of my Node assumptions are wrong: aren't there a huge
number of modules on NPM that don't require native compilation? Would
essentially every single Node project need support for native modules?

[tianon]

To clarify a little bit: the slim variants are intentionally supposed
to be for people who really know what they're doing and want to create
an image that's as small as is reasonably possible for their very
specific use-case.

[tianon]

(So if it's reasonable for a user to create a Node application that
doesn't require native modules, then IMO it's reasonable to not
include extra stuff to be able to build those modules here, and
instead document what's necessary to get there.)

[rvagg]

yes, that's a reasonable assumption, it's safe to say that if you are using compiled/native addons for Node.js then you're probably used to shaving a few yaks so it shouldn't a problem to have Docker images that don't support that out of the box, particularly if they are labelled "slim"

[retrohacker]

So to summarize where we are here, we want to remove python build-essential git and pkg-config from slim. The rationale is that not all Node applications require native modules, or running npm-install at all, and that the slim image should accommodate this use case.

Still exploring reducing the size of latest by:

• Depending on buildpack-deps:jessie-micro
• Cherry picking dependencies of build-essential for our needs as opposed to installing everything from build-essential

[retrohacker]

Will open a pull request with a revised version of slim for further comment.

[kriswill]

Here's a project that did something similar for old node.js https://github.com/jprjr/docker-tinynode. He got his image down to ~30 MB.

Then you can imagine a much faster on-boarding for demos:

→ docker run -it --rm iojs:tiny node -p process.versions
{ http_parser: '2.3.0',
  node: '1.1.0',
  v8: '4.1.0.14',
  uv: '1.3.0',
  zlib: '1.2.8',
  ares: '1.10.0-DEV',
  modules: '43',
  openssl: '1.0.1k' }

[CuthbertJungle]

instead of docker you could move to the unikernel http://osv.io/ it only includes files you need, there's a project for the old node.js code here: https://github.com/cloudius-systems/osv-apps/tree/master/node

[retrohacker]

Interesting output:

We have managed to cut 275 packages out of the final image for our slim variant.

[chorrell]

For Node.js some recent-ish work was done to trim down slim that you might want to incorporate:

nodejs/docker-node@181c14e

$ docker run -it node dpkg --get-selections | wc -l
     405
$ docker run -it node:slim dpkg --get-selections | wc -l
     114

[pesho]

@chorrell we considered doing the same in the past (#10 (comment)), but instead just switched to using buildpack-deps:jessie-curl as a base for slim (#19). It's a bit bigger (because of curl, wget, and ca-certificates), but simpler.

[retrohacker]

What is the rationale behind buildpack-deps:jessie-curl, I see that it saves a layer in our Dockerfile, but generally the argument for building off of buildpack-deps is to minimize the overall size of a docker installation by sharing as many layers between images as possible This contradicts the purpose of the slim image, which is generally used in single image deployments.

[pesho]

@wblankenship you did +1 #19, remember?

I guess the rationale is that it is simple, while still being almost minimal.

[retrohacker]

Yup, I do. I just want document our thought process so we can reference it moving forward 😄

[RnbWd]

I'm still using the original slim configs for iojs. It's only 289.8 MB, which 409.5 MB smaller than iojs:latest (699.3 MB) and 98.6 MB larger than iojs:slim (191.2 MB). The jessie equivalent would be 384.9 MB

I felt stuck between choosing a slim container that isn't compatible with any node-gyp repos vs. an enormous container that's feels bloated with unnecessary dependencies. I'm willing to sacrifice 98.6 MB for the those npm repo's, but not 409.5 MB.

Side Question: Is debian:jessie necessary for running iojs? Or is jessie mostly being used for the updated kernal and libraries. I haven't noticed any issues with wheezy - but all the official repos seem to have switched to jessie.

Edit: I had multiple libraries dependent on node:slim break a month or so ago without warning, which is why I'm maintaining this build. I know that 'slim' is meant for people that know what they're doing, but I'm not writing C++ modules, I'm just using npm. Unless I'm only using vanilla-JS npm-repo's, there's no reason to use iojs:slim when it's just as easy to build a container from debian's base image. There's no middle ground in the official repos (for iojs or node).

[retrohacker]

@RnbWd

So to summarize where we are here, we want to remove python build-essential git and pkg-config from slim. The rationale is that not all Node applications require native modules, or running npm-install at all, and that the slim image should accommodate this use case.

An easy way forward is to build off of the iojs:slim image and install these packages back on top in your personal repo, then add this repository as a repository link so your image rebuilds when ours does. You can certainly build from the debian base image, but the main advantage to building from this repo is that you will continue to get the latest versions of iojs when they release, with no extra effort on your part.

[RnbWd]

@wblankenship thanks - I think I'll do that. I'm hosting a node-io repo just for debian:wheezy slim. Although this is an issue I fault with node's official repo (which I began using before iojs) - this repo seems to copy their standards - but breaking slim (in node) was unexpected. Switching kernels and removing ggc support was just such a massive breaking change.

I'd love to help contribute a 'medium build' for those who don't want the full build-pack, but want the minimal gcc / python libs running out of the box :)

[RnbWd]

Actually after building, I noticed that using iojs:slim as a base created an image that's 415.5 MB - while the wheezy:slim version is 289.8 MB - so I'm sticking with wheezy. Maybe jessie's just bigger? But the goal is a slim image that also has python, git, and build essential - wheezy is definitely smaller base to use.

edit: I'm also exploring even smaller linux distros :)

[retrohacker]

@RnbWd if you have serious space constraints, take a look at building a static iojs executable in a volume in a throw away container (where you can have all your build dependencies installed). Then use an ADD to include that in your "real" image (maybe based on BusyBox). Just remember to keep libc dynamically linked, it is needed for some dns wizardry.

[RnbWd]

Thanks for the tip! My main goal is a minimal container that's capable of
downloading NPM repo's that use python / gcc. Running multiple docker
containers on a tiny cloud host can take up quite a lot of disk space, so
I'm trying to minimize that (and download IO time)

[pesho]

Closing as the original goals were achieved. Please open separate issues or PRs for new specific opportunities (such as #44).