Change Docker image to multi-stage build for Django

[arnav13081994]

Description

I am proposing to use multi-stage builds for building the python part of Django.

This can be done by installing all required OS and python dependencies and then to just copy over those files to the run stage. This reduces the size of a Django image from 509MB to 306MB (without gulp) and 561MB to 360MB (with gulp). A nearly 35% reduction in size!

One more improvement that can be made is to also copy over system dependencies from the python build stage, but that would make the configuration brittle due to apt installing files and symlinks in different locations depending on the package being installed. And that is why they are again installed in the run stage.

Checklist:

• I've made sure that tests/test_cookiecutter_generation.py is updated accordingly (especially if adding or updating a template option)
• I've updated the documentation or confirm that my change doesn't require any updates

Rationale

Smaller images are highly desirable because they would greatly reduce the time of re-deployment and deployment of containers and would also save a lot of time and money especially bandwidth costs.

Fix #3050

[demestav]

Wouldn't it be better to move the python requirements processing after the system requirements installation? Maybe I am wrong, but as you have it now, a change in a requirements file would force the system requirements to be installed since they are in a single command. I believe this is how it is currently implemented.

[arnav13081994]

@demestav Good catch. That would be a great thing to do in the build stage. We would be discarding the image layers in that stage anyway so no need to try to reduce the number of layers there. Made that change and pushed it.

[foarsitter]

Docker will cache tags like python:3.8-slim-buster without pulling a newer version when available. When testing your pull-request locally, the tag referenced to an earlier 3.8 build without having the correct pip installed. Therefore I recommend to pin the image to the exact version you are using.

[mwolff44]

I do not understand why you use wheel, as you need to install libq and do other steps in dockerfile. You can just copy libraries from builder steps ?
using wheel, also imposes to use the dockerfile in the same architecture

[arnav13081994]

@mwolff44 I tried doing that but even building wheels requires those dependencies to be installed. And the final run stage also needs them for postgres and django to run correctly that is why they are in 2 places.

[arnav13081994]

@browniebroke let me update this PR. I believe I had come up with a better way to write this multi stage dockerfile.

[arnav13081994]

@browniebroke I have updated both the files with the following changes:

1) Variables are used to indicate the `BUILD ENVIRONMENT` as well as the `PYTHON VERSION` which would make it super easy to upgrade python versions as well as change the build environment from local to production to staging etc.

2) Removed unnecessary cleanup steps in the BUILD STAGE because they wouldn't make it to the final image anyway.

3) Built Python wheels for every dependency and sub-dependency and then made pip install from only the created wheels. This would save a lot of time finding packages from pip for compatibility checks etc.

I would suggest building docker images by enabling docker buildkit as that allows one to also cache all apt and pip packages on the host system as well and can greatly speed up re-builds. All one would have to do is add the following line to the top of the dockerfile:

# syntax = docker/dockerfile:1.2.1 I can add those changes in a separate PR if that works better.

[browniebroke]

Looks good, tested a bit, works well 👍