add support pgbackrest

[vitabaks]

For bootstrap a patroni cluster and create replicas from backups.

• With support of point-in-time recovery, PITR.

pgBackRest User Guide: https://pgbackrest.org/user-guide-index.html
pgBackRest Configuration Reference: https://pgbackrest.org/configuration.html

• Installation pgbackrest on target servers (or update pgbackrest on repository host).
• Configure pgbackrest options on target servers.
• Generate and exchange of SSH Keys with pgbackrest server (Dedicated Repository Host).
• Bootstrap cluster using pgbackrest from Dedicated Repository Host (ssh).
• Create replicas using pgbackrest from Dedicated Repository Host (ssh).
• Bootstrap cluster using pgbackrest from S3 storage.
• Create replicas using pgbackrest from S3 storage.

[vitabaks]

To test for these changes, please clone the 'dev' branch:

git clone https://github.com/vitabaks/postgresql_cluster.git --branch dev

[vitabaks]

Added support of Point-In-Time Recovery, PITR (commits 874a074 , 9735c2f, 8c4b06c)

In the default mode (initdb), there is protection against deleting data from the database directory (PGDATA), and also there will be no attempt to remove the cluster from DCS.

But, if the variable "patroni_cluster_bootstrap_method" = "pgbackrest" or "wal-g":

1. the database cluster directory will be cleared (for "wal-g") or overwritten (for "pgbackrest" - delta restore).
2. and also the cluster "{{ patroni_cluster_name }}" will be deleted from DCS (if exist) before recovery.

requirements:
(will be installed automatically from pip repository if not exist)
pip3 install pexpect
pip3 install ruamel.yaml

For PITR only - run ansible with tag:

ansible-playbook deploy_pgcluster.yml --tags point_in_time_recovery

---

• Describe the details in the README.md file.

[github-actions]

yamllint Failed (fixed 603efbf)

Show Output

./roles/patroni/tasks/main.yml
  497:1     error    trailing spaces  (trailing-spaces)

Workflow: Yamllint, Action: karancodeyamllint-github-action, Lint: .

[vitabaks]

Tested with Debain and Ubuntu.

[vitabaks]

Tested with CentOS.

[vitabaks]

Tested with S3.
For testing, MinIO was used. Installed in accordance with this instructions.

example:
vars/main.yml

patroni_cluster_bootstrap_method: "pgbackrest"

patroni_create_replica_methods:
  - pgbackrest
  - basebackup

postgresql_restore_command: "pgbackrest --stanza={{ pgbackrest_stanza }} archive-get %f %p"

# pgBackRest
pgbackrest_install: true
pgbackrest_install_from_pgdg_repo: true
pgbackrest_stanza: "s3_stanza"
pgbackrest_repo_type: "s3"
pgbackrest_conf_file: "/etc/pgbackrest.conf"
pgbackrest_conf:
  global:  # [global] section
    - {option: "log-level-file", value: "detail"}
    - {option: "log-path", value: "/var/log/pgbackrest"}
    - {option: "repo1-type", value: "{{ pgbackrest_repo_type |lower }}"}
    - {option: "repo1-path", value: "/repo"}
    - {option: "repo1-s3-endpoint", value: "minio.local"}
    - {option: "repo1-s3-bucket", value: "pgbackrest"}
    - {option: "repo1-s3-verify-tls", value: "n"}
    - {option: "repo1-s3-key", value: "accessKey"}
    - {option: "repo1-s3-key-secret", value: "superSECRETkey"}
    - {option: "repo1-s3-region", value: "eu-west-3"}
    - {option: "repo1-retention-full", value: "4"}
    - {option: "start-fast", value: "y"}
    - {option: "delta", value: "y"}
  stanza:  # [stanza_name] section
    - {option: "pg1-path", value: "{{ postgresql_data_dir }}"}
    - {option: "process-max", value: "2"}
    - {option: "recovery-option", value: "recovery_target_action=promote"}
    - {option: "log-level-console", value: "info"}
pgbackrest_patroni_cluster_restore_command:
  '/usr/bin/pgbackrest --stanza={{ pgbackrest_stanza }} --delta restore'

vars/system.yml

etc_hosts:
  - "10.128.64.143 pgbackrest.minio.local minio.local s3.eu-west-3.amazonaws.com"

See more S3 repository options https://pgbackrest.org/configuration.html#section-repository