refactoring testing, building and Image system

This is a complete refactoring of our testing, building and Image system:
- no `./buildImages.sh` anymore it's all happening with `make`
- all images built by make, make also knows the dependency of the images and build them in the correct order
- docker-compose does not build images anymore, it just references images that are build by make
- new naming of images: version numbers all in image name, not in image tag (`centos7-node6` instead of `centos7-node:6`)
- no minishift anymore, we're using `oc cluster up` which runs within Docker, it's faster!
- using Travis for CI, which can run multiple tests at the same time.
- read DEVELOPMENT.md and TESTS.md to learn more

Author: Schnitzel

.dockerignore

@@ -1,2 +1,6 @@
 **/node_modules
-.dockerignore
+.dockerignore
+.git
+tests
+.cache
+Makefile

.gitignore

@@ -5,6 +5,11 @@ deploytest
 .DS_Store
 *.retry
 startVagrant.sh
+local-dev/oc
 minishift
 **/v8-*
 node_modules
+build/*
+!build/.gitkeep
+openshift
+.loopback

.travis.yml

@@ -0,0 +1,33 @@
+sudo: required
+services:
+- docker
+before_install:
+- curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
+- sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
+- sudo apt-get update
+- sudo apt-get -y install docker-ce
+- sudo sh -c "echo \"DOCKER_OPTS='--insecure-registry 172.30.0.0/16'\" >> /etc/default/docker" && sudo service docker restart && sudo mount --make-rshared /
+- docker info
+- docker version
+- docker login -u amazeeiojenkins -p $DOCKERHUB_PASSWORD
+install:
+- make pull -j6 TAG=$TRAVIS_BRANCH
+- docker image ls -a --digests
+before_script:
+- make openshift
+script:
+- docker logs -f origin &> .openshift-logs &
+- make build TAG=$TRAVIS_BRANCH
+- make tag-push TAG=$TRAVIS_BRANCH &> .push-logs &
+- make up
+- docker-compose -p lagoon logs -f &> .service-logs &
+- sleep 45
+- make tests
+after_script:
+- cat .build-logs
+- cat .openshift-logs
+- cat .push-logs
+- cat .service-logs
+notifications:
+  slack:
+    secure: XavhNWfkLB2MfuEg6UrIZ6jz3NNRr9drcyOWlI75wB2iPuP7C/rGrZ1NZJNjGvykM7JvZqcfD+tyzolP+MgzIthJF51mooC8QyfZc1BoiS2uhg+hD/g3COB+XbRnHv76Q4I42KRP/vHM2quNaW79VWovyCoTcWzJqVpKnLvs26asl38+5Z0oYGXTROC6c5rfow8aAsix1ZatCwugOQmVmepsKpLjNpiSv1OyWeIqURkqTxO2Q/Pw6Kow6O0DZZZW75swDNbNeC4OK4mZiGj5yPFRpiVqz1DHIWYAvCzkMVqgxubn5Ob/cDlkY306XGauetOlfx2SgL1FN4sbVFnu9t2JZJXj+Zzg7ccoGtN3rlfIpoUGb4U6hIQFmLky8XnZYnEF1D+n9ld3/y5WV7qOeVYTZsf5nEopD1Kq/8Zc+Hi6VJZ1nZEaovlLL2USnWfo4E/PlO3hSTF/9FEPy2JyWuExdcBxxTBWwgNwihBvpEyFG+YeryNR2kAlax8cRi8inhHLIBg+PhSDFB5r/kMjwDYqWQ/uTIFxLbZa+s1TEhNspPaeyDJkj8wYnlrlWuAC9fDppFFM8BbZrK1Lc6nIY0VMFVZOyKug6TW3/Jy2iKY1D0+EoCg78qHWn9xnXWfMugQKKLz+/p+WocC9HTiijYbqWtuUPBupF5SQpCQhw+0=

DEVELOPMENT.md

@@ -8,51 +8,55 @@ Please check the [official Docs of Docker](https://docs.docker.com/engine/instal
 
 ## Start Services
 
-1. clone this repo
+1. Add `172.30.0.0/16` to insecure registries in Docker (see [here](https://docs.docker.com/registry/insecure/) how to do that). Also make sure that you give your Docker Host minimum 4 CPUs and 6GB Ram. Also check that nothing is running on localhost:80 and localhost:443, as OpenShift will use these ports to run.
 
-2. build base images needed for testing
+2. Pull existing images for faster building as we use Docker Layer Caching
 
 ```sh
-./buildBaseImages.sh
+make pull
 ```
 
-3. start Lagoon Services
+Important: Lagoon consists of a lot of Services and Docker Images, building and running them locally might not even be necessary.
+We're using make (see the [Makefile](./Makefile)) in order to only build the needed Docker Images specifically for a part of Lagoon.
 
-```sh
-docker-compose up -d
-```
+All of it is based around tests. So if you like to only build the part that is needed to work on the Node.js deployment, you can run the tests with `make tests/node`, this will then setup (openshift, building images, services) all the needed stuff for the Node.js deployment part.
 
-4. Follow the Services logs
+If you would still like to build and start all services, go ahead:
+
+2. Build images
 
 ```sh
-docker-compose logs -f
+make build
 ```
 
-## Start & Test OpenShift
-
-1. start OpenShift
+3. start Lagoon Services
 
 ```sh
-./startOpenShift.sh
+make up
 ```
 
-2. Add `https://docker-registry-default.192.168.77.100.nip.io:443` to insecure registries in Docker (see [here](https://docs.docker.com/registry/insecure/) how to do that).
+4. Follow the Services logs
 
-4. run tests
+```sh
+make logs
+```
 
+5. run tests (read [TESTS.md](./TESTS.m) to learn more about testing)
 ```sh
-docker-compose run --rm tests ansible-playbook /ansible/tests/ALL.yaml
+make tests
 ```
 
+6. Look what happens in OpenShift: https://172.16.123.1:8443/console (developer/developer)
+
 ## Local Development
 
 Most services are written in Node.js. As many of these services share similar Node code and Node Packages, we're using a new feature of yarn, called `yarn workspaces`. Yarn Workspaces needs a package.json in the projects root directory that defines the workspaces plus an `.yarnrc` that enables workspace mode.
 
-The development of the services can happen directly within Docker. Each container for each service is setup in a way that it's source code is mounted into the running container. Node itself is watching the code via `nodemon` and restarts the node process automatically on a change.
+The development of the services can happen directly within Docker. Each container for each service is setup in a way that it's source code is mounted into the running container (see [docker-compose.yml](./docker-compose.yml). Node itself is watching the code via `nodemon` and restarts the node process automatically on a change.
 
 ### lagoon-commons
 
-The services not only share many node packages, but also share actual custom code. This code is within `node-packages/lagoon-commons` it will be automatically symlinked by yarn workspaces, plus the nodemon of the services is setup in a way that it also checks for changes in `node-packages` and will restart the node process automatically
+The services not only share many node packages, but also share actual custom code. This code is within `node-packages/lagoon-commons` it will be automatically symlinked by yarn workspaces, plus the nodemon of the services is setup in a way that it also checks for changes in `node-packages` and will restart the node process automatically.
 
 ### Hiera
 
@@ -62,10 +66,11 @@ The API uses a puppet compatible yaml format to store it's data. On production t
 
 **I can't build any docker image for any Node.js based service**
 
-Build the latest base images via
+Rebuild the images via
 
 ```sh
-./buildBaseImages.sh
+make clean
+make build
 ```
 
 **I get errors about missing node_modules content when I try to build / run a NodeJS based image**