-
Notifications
You must be signed in to change notification settings - Fork 57
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #113 from fullboar/feat/add-s3-support
feat: add s3 support and update documentation
- Loading branch information
Showing
10 changed files
with
206 additions
and
43 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -14,5 +14,7 @@ settings*.sh | |
|
|
||
| # Local config | ||
| .env | ||
| docker/backup.conf | ||
| backups | ||
| docker/backup.conf | ||
| minio-data | ||
| pg-data | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| ## Docker | ||
|
|
||
| This project contains a docker-compose.yml file suitable for development purposes. It will start a PostgreSQL database, minio (S3 compatible storage) and a backup container. | ||
|
|
||
| Because all the bash scripts are mounted to root `/` the docker-compose file mounts them all individually. If you wish to add new files, create it then add it to the backup container mounts. This method allows you to edit the files on the host and have changes immediately available in the container. You may need to restart the backup process to see the changes. | ||
|
|
||
| To start the containers run: | ||
|
|
||
| ```bash | ||
| docker-compose up | ||
| ``` | ||
|
|
||
| Once all 3 containers start they will create three folders that are **not** tracked by git: | ||
|
|
||
| - `./pg-data` - PostgreSQL database | ||
| - `./minio-data` - Minio storage | ||
| - `./backup` - Backup data | ||
|
|
||
| NOTE: Minio needs at minimum of 1G to function properly. Make sure you have some free local space. | ||
|
|
||
| Add this shell (bash or zsh) alias, it helps to quickly connect to a container by name: | ||
|
|
||
| ```bash | ||
| dcon='function _dcon(){ docker exec -i -t $1 /bin/bash -c "export COLUMNS=`tput cols`; export LINES=`tput lines`; exec bash"; };_dcon' | ||
| ``` | ||
|
|
||
| Use `docker ps` to list all running containers and then use `dcon <container_name>` to connect to the container. | ||
|
|
||
| ```bash | ||
| dcon backup-container-postgresql-1 | ||
| ``` | ||
|
|
||
| ### Protips 🤓 | ||
|
|
||
| - More modern version fo docker-compose do not require the hyphen. Try using `docker compose up`. | ||
| - This bash one-liner was used to add all the files to the backup container volumes `ls docker/ | grep 'backup\.' | awk '{ print "- ./docker/"$1":/"$1 }'` | ||
|
|
||
| ## Postgres | ||
|
|
||
| Connect to the postgres container using the `dcon` alias noted above and create the database, schema and then load in some sample data. | ||
|
|
||
| Connect to the postgres container: | ||
|
|
||
| ```bash | ||
| dcon backup-container-postgresql-1 | ||
| ``` | ||
|
|
||
| Run the `psql` command to connect to the database and create the database: | ||
|
|
||
| ```bash | ||
| psql -U postgres | ||
| create database sakila; | ||
| ^D | ||
| ``` | ||
|
|
||
| Run the following two commands to create the schema and insert the data. If `curl` is not installed run `apt-get update && apt-get install curl` first. | ||
|
|
||
| ```bash | ||
| curl -sSL https://raw.githubusercontent.com/jOOQ/sakila/main/postgres-sakila-db/postgres-sakila-schema.sql | psql -U postgres -d sakila; | ||
| ``` | ||
|
|
||
| ```bash | ||
| curl -sSL https://raw.githubusercontent.com/jOOQ/sakila/main/postgres-sakila-db/postgres-sakila-insert-data.sql | psql -U postgres -d sakila; | ||
| ``` | ||
|
|
||
| You should now have a sample database with data. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| postgres=postgresql:5432/sakila | ||
| 1 1 * * * default ./backup.sh -s | ||
| 1 4 * * * default ./backup.sh -s -v all |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| version: '3.8' | ||
| services: | ||
| minio: | ||
| image: minio/minio | ||
| volumes: | ||
| - ./minio-data:/data | ||
| ports: | ||
| - 9000:9000 | ||
| - 9001:9001 | ||
| environment: | ||
| MINIO_ROOT_USER: minio | ||
| MINIO_ROOT_PASSWORD: secret00 | ||
| command: server /data --console-address ":9001" | ||
| postgresql: | ||
| image: postgres:12 | ||
| volumes: | ||
| - ./pg-data:/var/lib/postgresql/data | ||
| ports: | ||
| - 5432:5432 | ||
| environment: | ||
| POSTGRES_USER: postgres | ||
| POSTGRES_PASSWORD: secret | ||
| backup: | ||
| build: ./docker | ||
| environment: | ||
| DATABASE_USER: postgres | ||
| DATABASE_PASSWORD: secret | ||
| BACKUP_CONF: /conf/backup.conf | ||
| S3_USER: minio | ||
| S3_PASSWORD: secret00 | ||
| S3_ENDPOINT: http://minio:9000 | ||
| S3_BUCKET: backups | ||
| volumes: | ||
| - ./backups:/backups | ||
| - ./config/backup-dev.conf:/conf/backup.conf | ||
| - ./docker/backup.config.utils:/backup.config.utils | ||
| - ./docker/backup.container.utils:/backup.container.utils | ||
| - ./docker/backup.file.utils:/backup.file.utils | ||
| - ./docker/backup.ftp:/backup.ftp | ||
| - ./docker/backup.logging:/backup.logging | ||
| - ./docker/backup.mariadb.plugin:/backup.mariadb.plugin | ||
| - ./docker/backup.misc.utils:/backup.misc.utils | ||
| - ./docker/backup.mongo.plugin:/backup.mongo.plugin | ||
| - ./docker/backup.mssql.plugin:/backup.mssql.plugin | ||
| - ./docker/backup.null.plugin:/backup.null.plugin | ||
| - ./docker/backup.postgres.plugin:/backup.postgres.plugin | ||
| - ./docker/backup.server.utils:/backup.server.utils | ||
| - ./docker/backup.settings:/backup.settings | ||
| - ./docker/backup.sh:/backup.sh | ||
| - ./docker/backup.usage:/backup.usage | ||
| - ./docker/backup.utils:/backup.utils | ||
| - ./docker/backup.s3:/backup.s3 |
Oops, something went wrong.