Initial release (#1)

* Initial local-only WIP implementation

* Update Dockerfile to work within Triton "docker build"

* Adjust CONSUL to be either inferred to be "consul" or pulled from _env (so that Triton CNS can be used on Triton)

* Add initial "docker-compose.yml" for deploying on Triton

* Update README with more details of how to run/use this (and what it is)

* Add note about Mongo Express configuration parameters

* Update ContainerPilot

* Update Consul

* Add Triton setup script for generating initial _env, with CNS consul hostname

* Update README to reflect setup.sh

* backtick _env

.dockerignore

@@ -0,0 +1,3 @@
+.DS_Store
+**/.DS_Store
+_env*

.gitignore

@@ -0,0 +1,13 @@
+# build outputs
+*.pyc
+
+# credentials
+_env*
+manta
+manta.pub
+
+# temp
+python-manta/
+
+# macos frustration
+.DS_Store

Dockerfile

@@ -0,0 +1,42 @@
+FROM mongo-express:0.32
+
+# we need Container Pilot to monitor consul for MongoDB cluster changes so we can update ME_CONFIG_MONGODB_SERVER appropriately
+# "If it is a replica set, use a comma delimited list of the host names."
+
+RUN apt-get update \
+	&& apt-get install -y \
+		ca-certificates \
+		unzip \
+		wget \
+	&& rm -rf /var/lib/apt/lists/*
+
+ENV CONTAINERPILOT_VERSION 2.7.0
+ENV CONTAINERPILOT_SHA1 687f7d83e031be7f497ffa94b234251270aee75b
+RUN set -x \
+	&& wget -O containerpilot.tar.gz "https://github.com/joyent/containerpilot/releases/download/${CONTAINERPILOT_VERSION}/containerpilot-${CONTAINERPILOT_VERSION}.tar.gz" \
+	&& echo "${CONTAINERPILOT_SHA1} *containerpilot.tar.gz" | sha1sum -c - \
+	&& tar -xvf containerpilot.tar.gz -C /usr/local/bin \
+	&& rm -v containerpilot.tar.gz
+
+ENV CONSUL_TEMPLATE_VERSION 0.16.0
+ENV CONSUL_TEMPLATE_SHA256 064b0b492bb7ca3663811d297436a4bbf3226de706d2b76adade7021cd22e156
+RUN set -x \
+	&& wget -O consul-template.zip "https://releases.hashicorp.com/consul-template/${CONSUL_TEMPLATE_VERSION}/consul-template_${CONSUL_TEMPLATE_VERSION}_linux_amd64.zip" \
+	&& echo "${CONSUL_TEMPLATE_SHA256} *consul-template.zip" | sha256sum -c - \
+	&& unzip consul-template.zip -d /usr/local/bin \
+	&& rm -v consul-template.zip
+
+ENV CONTAINERPILOT file:///etc/containerpilot.json
+
+COPY etc/* /etc/
+COPY bin/* /usr/local/bin/
+
+# override any parent entrypoint
+ENTRYPOINT []
+# see comment in bin/mongo-express-wrapper.sh for why it is necessary
+# "node app" is taken from the "mongo-express" default CMD (no "tini" since we have containerpilot performing those duties instead)
+CMD [ \
+	"containerpilot", \
+	"mongo-express-wrapper.sh", \
+	"node", "app" \
+]

README.md

@@ -1,2 +1,41 @@
-# mongo-express
-Work in progress, not stable, expect force pushes of this repo
+# AutoPilot Pattern Mongo Express
+
+*An example implementation of a simple MongoDB application ([Mongo Express](https://github.com/mongo-express/mongo-express)) on top of [AutoPilot Pattern MongoDB](https://github.com/autopilotpattern/mongodb)*
+
+## Architecture
+
+A running cluster includes the following components:
+- [AutoPilot Pattern MongoDB](https://github.com/autopilotpattern/mongodb): for auto-scaling MongoDB via replica sets
+- [ContainerPilot](https://www.joyent.com/containerpilot): included in our Mongo Express container to coordinate runtime configuration rewriting after cluster changes
+- [Consul](https://www.consul.io/): used to coordinate replication and failover
+- [Mongo Express](https://github.com/mongo-express/mongo-express): a simple MongoDB application for browsing/editing the data stored in our cluster
+
+## Running the cluster
+
+Starting a new cluster is easy once you have [your `_env` file set with the configuration details](#configuration)
+
+**Triton**
+- `./setup.sh`
+- `docker-compose up -d`
+- `open $(triton ip mongoexpress_mongo-express_1):8081`
+
+**Local (or Non-Triton)**
+- `docker-compose up -f local-compose.yml -d`
+- `open localhost:<assigned port>`
+
+The `setup.sh` script will check that your environment is setup correctly and will create an `_env` file that includes injecting an environment variable for the Consul hostname into the Mongo and Mongo-Express containers so we can take advantage of [Triton Container Name Service (CNS)](https://www.joyent.com/blog/introducing-triton-container-name-service).
+
+The MongoDB cluster will be initialized according to the process outlined in the [AutoPilot Pattern MongoDB README](https://github.com/autopilotpattern/mongodb/tree/master/README.md), and Mongo Express will be configured to point at all active MongoDB nodes in the cluster (via Consul and ContainerPilot).
+
+**Run `docker-compose -f local-compose.yml scale mongodb=2` to add a replica (or more than one!)**. The replicas will automatically be added to the replica set on the master and will register themselves in Consul as replicas once they're ready, and the Mongo Express configuration will be reloaded to point to the new replicas.
+
+### Configuration
+
+The [configuration items noted for AutoPilot Pattern MongoDB](https://github.com/autopilotpattern/mongodb#configuration) apply for this image as well as the following explicit parameters for this implementation (passed in via `_env` file next to `docker-compose.yml` and `local-compose.yml`):
+
+- `CONSUL` (optional): when using `local-compose.yml`, this will default to `consul` (and thus use the DNS provided by Docker), but for deploying on Triton via `docker-compose.yml`, this should be set to [the CNS path of the `consul` service (`consul.svc.XXX...`)](https://docs.joyent.com/public-cloud/network/cns)
+- any Mongo Express environment variables, such as `ME_CONFIG_OPTIONS_EDITORTHEME` or `ME_CONFIG_MONGODB_ENABLE_ADMIN`; see [the Mongo Express documentation](https://www.npmjs.com/package/mongo-express#usage-docker) for a more complete list
+
+### Sponsors
+
+Initial development of this project was sponsored by [Joyent](https://www.joyent.com).

bin/mongo-express-wrapper.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+set -e
+
+# we can't inject new environment variables directly into Container Pilot, so instead we update/source a file at startup which our co-processes can modify for us
+if [ -f /etc/mongo-express-env ]; then
+	. /etc/mongo-express-env
+fi
+
+exec "$@"

docker-compose.yml

@@ -0,0 +1,28 @@
+version: '2'
+
+services:
+
+    mongo-express:
+        extends:
+            file: local-compose.yml
+            service: mongo-express
+        labels:
+            - triton.cns.services=mongo-express
+        network_mode: bridge
+
+    mongodb:
+        extends:
+            file: local-compose.yml
+            service: mongodb
+        mem_limit: 4g
+        labels:
+            - triton.cns.services=mongod
+        network_mode: bridge
+
+    consul:
+        extends:
+            file: local-compose.yml
+            service: consul
+        labels:
+            - triton.cns.services=consul
+        network_mode: bridge

etc/containerpilot.json

@@ -0,0 +1,30 @@
+{
+	"consul": "{{ if .CONSUL }}{{ .CONSUL }}{{ else }}consul{{ end }}:8500",
+	"preStart": [
+		"consul-template",
+		"-once",
+		"-consul", "{{ if .CONSUL }}{{ .CONSUL }}{{ else }}consul{{ end }}:8500",
+		"-template", "/etc/mongo-express-env.ctmpl:/etc/mongo-express-env"
+	],
+	"services": [
+		{
+			"name": "mongo-express",
+			"port": 8081,
+			"health": "wget -q -O /dev/null http://localhost:8081",
+			"poll": 5,
+			"ttl": 25
+		}
+	],
+	"backends": [
+		{
+			"name": "mongodb-replicaset",
+			"poll": 5,
+			"onChange": [
+				"consul-template",
+				"-once",
+				"-consul", "{{ if .CONSUL }}{{ .CONSUL }}{{ else }}consul{{ end }}:8500",
+				"-template", "/etc/mongo-express-env.ctmpl:/etc/mongo-express-env:kill -TERM 1"
+			]
+		}
+	]
+}

etc/mongo-express-env.ctmpl

@@ -0,0 +1,8 @@
+{{- if service "mongodb-replicaset" -}}
+export ME_CONFIG_MONGODB_SERVER="
+	{{- range $i, $e := service "mongodb-replicaset" -}}
+		{{- if ne $i 0 -}} , {{- end -}}
+		{{ $e.Address }}
+	{{- end -}}
+"
+{{ end -}}

local-compose.yml

@@ -0,0 +1,40 @@
+version: '2'
+
+services:
+
+    mongo-express:
+        image: autopilotpattern/mongo-express
+        build: .
+        restart: always
+        mem_limit: 512m
+        env_file: _env
+        environment:
+            - ME_CONFIG_OPTIONS_EDITORTHEME=ambiance
+# ME_CONFIG_MONGODB_SERVER will get re-written by consul-template
+            - ME_CONFIG_MONGODB_SERVER=mongodb
+        ports:
+            - 8081
+
+    mongodb:
+        image: autopilotpattern/mongodb
+        command: containerpilot mongod --replSet=pilot
+        restart: always
+        mem_limit: 512m
+        env_file: _env
+        ports:
+            - 27017
+
+    consul:
+        image: consul:0.7.2
+        command: agent -dev -ui -client=0.0.0.0
+        restart: always
+        mem_limit: 128m
+        ports:
+            - 8500
+        expose:
+            - 53
+            - 8300
+            - 8301
+            - 8302
+            - 8400
+            - 8500

setup.sh

@@ -0,0 +1,133 @@
+#!/bin/bash
+set -e -o pipefail
+
+help() {
+    echo 'Usage ./setup.sh [-f docker-compose.yml] [-p project]'
+    echo
+    echo 'Checks that your Triton and Docker environment is sane and configures'
+    echo 'an environment file to use.'
+    echo
+    echo 'Optional flags:'
+    echo '  -f <filename>   use this file as the docker-compose config file'
+    echo '  -p <project>    use this name as the project prefix for docker-compose'
+}
+
+
+# default values which can be overriden by -f or -p flags
+export COMPOSE_PROJECT_NAME=mongo-express
+export COMPOSE_FILE=
+
+# give the docker remote api more time before timeout
+export COMPOSE_HTTP_TIMEOUT=300
+
+# populated by `check` function whenever we're using Triton
+TRITON_USER=
+TRITON_DC=
+TRITON_ACCOUNT=
+
+# ---------------------------------------------------
+# Top-level commmands
+
+
+# Check for correct configuration
+check() {
+
+    command -v docker >/dev/null 2>&1 || {
+        echo
+        tput rev  # reverse
+        tput bold # bold
+        echo 'Docker is required, but does not appear to be installed.'
+        tput sgr0 # clear
+        echo 'See https://docs.joyent.com/public-cloud/api-access/docker'
+        exit 1
+    }
+    command -v json >/dev/null 2>&1 || {
+        echo
+        tput rev  # reverse
+        tput bold # bold
+        echo 'Error! JSON CLI tool is required, but does not appear to be installed.'
+        tput sgr0 # clear
+        echo 'See https://apidocs.joyent.com/cloudapi/#getting-started'
+        exit 1
+    }
+
+    # if we're not testing on Triton, don't bother checking Triton config
+    if [ ! -z "${COMPOSE_FILE}" ]; then
+        exit 0
+    fi
+
+    command -v triton >/dev/null 2>&1 || {
+        echo
+        tput rev  # reverse
+        tput bold # bold
+        echo 'Error! Joyent Triton CLI is required, but does not appear to be installed.'
+        tput sgr0 # clear
+        echo 'See https://www.joyent.com/blog/introducing-the-triton-command-line-tool'
+        exit 1
+    }
+
+    # make sure Docker client is pointed to the same place as the Triton client
+    local docker_user=$(docker info 2>&1 | awk -F": " '/SDCAccount:/{print $2}')
+    local docker_dc=$(echo $DOCKER_HOST | awk -F"/" '{print $3}' | awk -F'.' '{print $1}')
+    TRITON_USER=$(triton profile get | awk -F": " '/account:/{print $2}')
+    TRITON_DC=$(triton profile get | awk -F"/" '/url:/{print $3}' | awk -F'.' '{print $1}')
+    TRITON_ACCOUNT=$(triton account get | awk -F": " '/id:/{print $2}')
+    if [ ! "$docker_user" = "$TRITON_USER" ] || [ ! "$docker_dc" = "$TRITON_DC" ]; then
+        echo
+        tput rev  # reverse
+        tput bold # bold
+        echo 'Error! The Triton CLI configuration does not match the Docker CLI configuration.'
+        tput sgr0 # clear
+        echo
+        echo "Docker user: ${docker_user}"
+        echo "Triton user: ${TRITON_USER}"
+        echo "Docker data center: ${docker_dc}"
+        echo "Triton data center: ${TRITON_DC}"
+        exit 1
+    fi
+
+    local triton_cns_enabled=$(triton account get | awk -F": " '/cns/{print $2}')
+    if [ ! "true" == "$triton_cns_enabled" ]; then
+        echo
+        tput rev  # reverse
+        tput bold # bold
+        echo 'Error! Triton CNS is required and not enabled.'
+        tput sgr0 # clear
+        echo
+        exit 1
+    fi
+
+    echo CONSUL=consul.svc.${TRITON_ACCOUNT}.${TRITON_DC}.cns.joyent.com > _env
+    echo "MONGO_SECONDARY_CATCHUP_PERIOD=60" >> _env
+    echo "MONGO_STEPDOWN_TIME=5" >> _env
+    echo "MONGO_ELECTION_TIMEOUT=60" >> _env
+
+}
+
+# ---------------------------------------------------
+# parse arguments
+
+while getopts "f:p:h" optchar; do
+    case "${optchar}" in
+        f) export COMPOSE_FILE=${OPTARG} ;;
+        p) export COMPOSE_PROJECT_NAME=${OPTARG} ;;
+    esac
+done
+shift $(expr $OPTIND - 1 )
+
+until
+    cmd=$1
+    if [ ! -z "$cmd" ]; then
+        shift 1
+        $cmd "$@"
+        if [ $? == 127 ]; then
+            help
+        fi
+        exit
+    fi
+do
+    echo
+done
+
+# default behavior
+check