Skip to content

Deployment: Quickstart with Docker

Jump to bottom
Edwin Guzman edited this page Apr 28, 2021 · 36 revisions

So you're deploying your library's circulation manager. Awesome! If you'd like to get up and running quickly, we recommend using our Docker image.

Contents:

Prep Work

  1. Install Docker. Docker has step-by-step instructions to grab its most up-to-date version. Depending on your package manager, you could also install a slightly older version with: sudo apt-get install docker-ce or sudo yum install docker-ce.

  2. Create any dependent, temporary containers (optional) for integrations like Elasticsearch and Postgres. We don't recommend using containers in the long-term for holding or maintaining data. However, if you just want to get a sense of how your Circulation Manager will work, containers are a quick option. Instructions for integrating Elasticsearch and Postgres via Docker can be found below.

  3. Get the Docker images for the Library Simplified Circulation Manager. Run:

    $ sudo docker pull nypl/circ-webapp && sudo docker pull nypl/circ-scripts

Running Circulation Manager containers

Running scripts

To deploy an app filled with your library's books, you'll need to run a number of scripts. Read the environment variable details below before running this script; you will likely need to alter it to meet your needs.

Example docker run script

$ sudo docker run -d --name circ-scripts \
    -e TZ="US/Central" \
    -e SIMPLIFIED_DB_TASK='init' \
    -e SIMPLIFIED_PRODUCTION_DATABASE='postgres://[username]:[password]@[host]:[port]/[database_name]' \
    nypl/circ-scripts

What It Does

The example above runs this resulting container in detached mode (-d), linked to the (-e) SIMPLIFIED_PRODUCTION_DATABASE and calling it "circ-scripts". With the (-e) optional argument TZ, you can pass a Debian-system timezone representing your local time zone, which will cause timed scripts to run according to your local time. If the database you've connected in your configuration has never been used before, use -e to set the optional argument SIMPLIFIED_DB_TASK to 'init'. This will keep track of the state of the database you've created and create an alias on your Elasticsearch cluster, allowing database updates to be easily managed with scripts.

Running Scripts

Once you've given your scripts some time to run (~30 minutes should be enough time to start having works move through the import process), you'll want to refresh your cached materialized views so they show up in your deployed app.

```sh
$ sudo docker exec circ-scripts /var/www/circulation/core/bin/run refresh_materialized_views
```

Troubleshooting

You'll want to check the logs of your container. For example:

# check logs of the database task and running supervisor processes
$ sudo docker logs circ-scripts

# check logs of cron and scripts
$ sudo docker exec circ-scripts cat /var/log/cron.log | less
$ sudo docker exec circ-scripts ls /var/log/simplified
$ sudo docker exec circ-scripts cat /var/log/simplified/overdrive_monitor_full | less

# The log directory can also be found on the production server.
# Its location can be found using this command.
$ sudo docker inspect circ-scripts \
  --format='{{range $mount := .Mounts}}{{if eq $mount.Destination "/var/log"}}{{$mount.Source}}{{end}}{{end}}'

You can hop into a running container at any time with the command: $ sudo docker exec -it circ /bin/bash

Docker has fantastic documentation to get more familiar with its command line tools, like docker exec and docker inspect. We recommend you check them out.

Deploying the App

Using an nypl/circ-webapp container deploys the OPDS feeds expected by the SimplyE client applications. Read the environment variable details below before running the following script; you will likely need to alter it to meet your needs.

Example docker run script

$ sudo docker run --name circ-webapp \
    -d -p 80:80 \
    -e SIMPLIFIED_PRODUCTION_DATABASE='postgres://[username]:[password]@[host]:[port]/[database_name]' \
    -e SIMPLIFIED_DB_TASK="migrate" \
    nypl/circ-webapp

What It Does

The script above runs the container in detached mode (-d), binding its port 80 to your server's port 80 (-p), connecting it to your PostgreSQL database -e SIMPLIFIED_PRODUCTION_DATABASE) and calling it "circ-webapp". Unless you've been running a scripts container for while, when you visit your server through a browser, you'll see a very sparse OPDS feed. If the database you've connected in your configuration has never been used before, use -e to set the optional argument SIMPLIFIED_DB_TASK to 'init'. This will keep track of the state of the database you've created and create an alias on your Elasticsearch cluster, allowing database updates to be easily managed with scripts.

Troubleshooting

You'll want to check the logs of your container (/var/log/nginx/error.log and /var/log/uwsgi/uwsgi.log) to troubleshoot:

# check logs of the database task and running supervisor processes
$ sudo docker logs circ-webapp

# check logs inside the container
$ sudo docker exec circ-webapp cat /var/log/nginx/error.log | less
$ sudo docker exec circ-webapp cat /var/log/uwsgi/uwsgi.log | less

# restart the application
$ sudo docker exec circ-webapp touch uwsgi.ini

You can hop into a running container at any time with the command: $ sudo docker exec -it circ /bin/bash

Docker has fantastic documentation to get more familiar with its command line tools, like docker exec and docker inspect. We recommend you check them out.

Environment Variables

SIMPLIFIED_CONFIGURATION_FILE

Optional The full path to configuration file in the container. Using the volume options -v, e.g. YOUR_LOCAL_DIRECTORY_WITH_CONFIG_FILE:/etc/simplified, it should look something like -e SIMPLIFIED_CONFIGURATION_FILE=/etc/simplified/YOUR_CONFIGURATION_FILENAME.json.

Use this documentation to create the JSON file for your particular library's configuration. If you're unfamiliar with JSON, you can use this JSON Formatter & Validator to validate your configuration file.

SIMPLIFIED_DB_TASK

Required. Performs a task against the database at container runtime. Options are:

  • ignore : Does nothing. This is the default value.
  • init : Initializes the app against a brand new database. If you are running a circulation manager for the first time every, use this value to set up an Elasticsearch alias and account for the database schema for future migrations.
  • migrate : Migrates an existing database against a new release. Use this value when switching from one stable version to another.

SIMPLIFIED_PRODUCTION_DATABASE

Required. The URL of the production PostgreSQL database for the application.

SIMPLIFIED_TEST_DATABASE

Optional in v2.x only. The URL of a PostgreSQL database for tests. This optional variable allows unit tests to be run in the container.

TZ

Optional. Scripts container only. The timezone of the library or libraries on this circulation manager, selected according to Debian-system timezone options. This value allows scripts to run at ideal times.

Evaluating Success

If your Docker containers are running successfully, you should have a /var/log/simplified directory full of logfiles in your circ-scripts container, and you should be able to visit your server's domain and see an OPDS feed from circ-webapp. If either of these things aren't occurring, use the troubleshooting details above to check var/log/cron.log or the logfiles in /var/log/simplified for circ-scripts and/or /var/log/uwsgi/uwsgi.log or /var/log/nginx/error.log.

Support Containers (for use in development or testing)

Elasticsearch

While we do not recommend you run Elasticsearch from a Docker container permanently, you may want to get up and running with a throwaway search index. Elasticsearch isn't installed via the Dockerfile, so the fastest way to connect to it will be through another container. Here's how:

  1. Get the Docker image for Elasticsearch v1.x:

    $ sudo docker pull elasticsearch:1

  2. Create an Elasticsearch container, and grab its IP Address. Run:

    $ sudo docker run -d --name es elasticsearch:1     # create an elasticsearch container
$ sudo docker ps                                   # confirm that it's running
# note its IP address
$ sudo docker inspect es --format="{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}"

  3. Add the Elasticsearch URL to your configuration file. When you run sudo docker ps, you'll see a single running container called es. Use the IP that comes from running inspect to update your your config.json file with the proper Elasticsearch location. You should end up with something like "http://172.17.0.2:9200".

Postgres

While we do not recommend you run Postgres from a Docker container permanently, you may want to get up and running with a throwaway database. Postgres isn't installed via the Dockerfile, so the best way to connect to Postgres will be through another container. Here's how:

  1. Get the Docker image for Postgres 12:

    $ sudo docker pull postgres:12

  2. Create a Postgres container, and grab its IP Address. Run:

    $ sudo docker run -d --name pg postgres:12        # create a postgres container
$ sudo docker ps                                   # confirm that it's running
# note its IP address
$ sudo docker inspect pg --format="{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}"

  3. Create a Postgres database. Run:

    $ docker exec -u postgres pg psql -c "create user simplified with password 'test';"    # create a user and password
$ docker exec -u postgres pg psql -c "create database simplified_circ_db;"            # create database
$ docker exec -u postgres pg psql -c "grant all privileges on database simplified_circ_db to simplified;"
$ docker exec -u postgres pg psql -d simplified_circ_db -c "create extension pgcrypto;"

  4. Add the Postgres URL to your configuration file. In config.json, add the appropriate production_url. You should end up with something like "postgres://simplified:[email protected]:5432/simplified_circ_db", following the "postgres://<USERNAME>:<PASSWORD>@<HOST>:<PORT>/<DATABASE_NAME>" format.

Clone this wiki locally