Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add create global cluster with rds cluster role #83

Merged
merged 41 commits into from
Oct 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
a9eca59
add create global cluster role
taehopark32 Jul 28, 2023
2ebf606
added test cases
taehopark32 Jul 31, 2023
c401f73
fixed some issues
taehopark32 Aug 4, 2023
963bfb2
rename role, minor fixes
mandar242 Aug 10, 2023
218c94b
restructure create_rds_global_cluster role
mandar242 Aug 11, 2023
d120444
update readme
mandar242 Aug 11, 2023
ea16e03
update readme
mandar242 Aug 11, 2023
1d19c6b
updates to tests
mandar242 Aug 15, 2023
7c02c81
add newline at EOF
mandar242 Aug 15, 2023
e8a23dc
update tests
mandar242 Aug 16, 2023
8ab2d34
add example playbook to readme
mandar242 Aug 29, 2023
408a659
add code to handle deleting created infra
mandar242 Sep 30, 2023
79490b1
add example for `delete`
mandar242 Sep 30, 2023
97e4c3b
fix tests
mandar242 Sep 30, 2023
daba80e
global cluster engine fix
mandar242 Oct 3, 2023
ff3206c
minor fixes
mandar242 Oct 3, 2023
541082d
replace manual cleanup tasks with role specific deletion task
mandar242 Oct 3, 2023
8ba1e02
minor fixes
mandar242 Oct 3, 2023
64138b3
make engine_version mandatory
mandar242 Oct 3, 2023
5819fd5
add support for db_name, enable_global_write_forwarding
mandar242 Oct 3, 2023
5308bf8
minor fixes, rename username, password to master_username, master_use…
mandar242 Oct 4, 2023
106fcbf
add support for vpc_security_group_ids
mandar242 Oct 4, 2023
21b2a8e
make linter happy
mandar242 Oct 4, 2023
7bce7cf
README fixes
mandar242 Oct 9, 2023
28aa491
remove extra vars in delete operation
mandar242 Oct 10, 2023
102b9b8
remove comments
mandar242 Oct 10, 2023
afe1dab
add changelogs fragment
mandar242 Oct 10, 2023
fcaef26
update readme
mandar242 Oct 10, 2023
66bf548
linter fixes
mandar242 Oct 11, 2023
04f5b37
linter fixes
mandar242 Oct 11, 2023
4652256
linter fixes
mandar242 Oct 11, 2023
d4d7ee7
linter fixes
mandar242 Oct 11, 2023
3b35b5c
update based on review feedback
mandar242 Oct 11, 2023
c4dc368
add module defaults for amazon.cloud
mandar242 Oct 12, 2023
791a138
add default values
mandar242 Oct 16, 2023
d4c3ecc
add default values
mandar242 Oct 16, 2023
6af9d3c
sanity fix
mandar242 Oct 16, 2023
91c6505
update role to use `aws_region` instead of `region`
mandar242 Oct 17, 2023
4ce30c8
rename default vars to remove default filter use
mandar242 Oct 17, 2023
da8773f
rename default vars to remove default filter use
mandar242 Oct 17, 2023
b718282
add default value for engine version
mandar242 Oct 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ Name | Description
[cloud.aws_ops.awsconfig_apigateway_with_lambda_integration](https://github.com/ansible-collections/cloud.aws_ops/blob/main/roles/awsconfig_apigateway_with_lambda_integration/README.md)|A role to create/delete an API gateway with lambda function integration.
[cloud.aws_ops.manage_transit_gateway](https://github.com/ansible-collections/cloud.aws_ops/blob/main/roles/manage_transit_gateway/README.md)|A role to create/delete transit_gateway with vpc and vpn attachments.
[cloud.aws_ops.deploy_flask_app](https://github.com/ansible-collections/cloud.aws_ops/blob/main/roles/deploy_flask_app/README.md)|A role to deploy a flask web application on AWS.
[cloud.aws_ops.create_rds_global_cluster](https://github.com/ansible-collections/cloud.aws_ops/blob/main/roles/create_rds_global_cluster/README.md)|A role to create, delete aurora global cluster with a primary cluster and a replica cluster in different regions.

### Playbooks
Name | Description
Expand Down
3 changes: 3 additions & 0 deletions changelogs/fragments/create_rds_global_cluster_role.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
---
minor_changes:
- create_rds_global_cluster - new role to create aurora global cluster with a primary and a replica cluster in different regions.
1 change: 1 addition & 0 deletions galaxy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ tags:
dependencies:
amazon.aws: '>=5.1.0'
community.aws: '>=5.0.0'
amazon.cloud: '>=0.4.0'
version: 1.0.3
build_ignore:
- .DS_Store
Expand Down
107 changes: 107 additions & 0 deletions roles/create_rds_global_cluster/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
create_rds_global_cluster
=========

A role to create an Amazon Aurora global cluster with two different region rds clusters.

Creates the following resources:
1. Global Cluster - Amazon Aurora Postgresql or Amazon Aurora MySql cluster. If `create_rds_global_cluster_engine` is not provided, defaults to Amazon Aurora Postgresql.
2. Primary Cluster - Primary cluster in specified region (`create_rds_global_cluster_primary_cluster_region`).
3. Primary Cluster Instance - Instance in the primary cluster.
4. Replica (secondary) Cluster - Secondary cluster in specified region (`create_rds_global_cluster_replica_cluster_region`).
5. Replica Cluster Instance - Instance in the replica cluster.

Please refer to [Role Variables](#role-variables) for variables and usage.

Requirements
------------

AWS User Account with the following permissions:

* rds:CreateGlobalCluster
* rds:DeleteGlobalCluster
* rds:ModifyGlobalCluster
* rds:CreateDBCluster
* rds:DeleteDBCluster
* rds:ModifyDBCluster
* rds:DescribeGlobalClusters
* rds:DescribeDBClusters

Role Variables
--------------
**Global cluster variables**
- **create_rds_global_cluster_global_cluster_name** - Name of the Amazon Aurora global cluster. **required**
- **create_rds_global_cluster_engine** - Engine of the Amazon Aurora global and rds clusters. Default is aurora-postgresql.
- **create_rds_global_cluster_engine_version** - Engine version of the Amazon Aurora global and rds clusters.
- **create_rds_global_cluster_instance_class** - Instance class of instance in primary and replica cluster. **required**
- **create_rds_global_cluster_master_username** - Username of the rds clusters master user. **required**
- **create_rds_global_cluster_master_user_password** - Password of the rds clusters master user. **required**

**Primary cluster variables**
- **create_rds_global_cluster_primary_cluster_name** - Name of the primary cluster. Default is $create_rds_global_cluster_global_cluster_name.
- **create_rds_global_cluster_primary_cluster_region** - Region of the primary cluster. **required**
- **create_rds_global_cluster_primary_cluster_instance_name** - Name of the instance in primary cluster. **required**
- **create_rds_global_cluster_primary_cluster_db_name** - The name for your database of up to 64 alphanumeric characters. If not provided, database is not created in the cluster.
- **create_rds_global_cluster_primary_cluster_vpc_security_group_ids** - A list of EC2 VPC security groups to associate with the primary DB cluster.
mandar242 marked this conversation as resolved.
Show resolved Hide resolved
- **create_rds_global_cluster_db_subnet_group_name** - A DB subnet group to associate with this DB cluster if not using the default.

**Replica cluster variables**
- **create_rds_global_cluster_replica_cluster_name** - Name of the replica (secondary) cluster. Default is create_rds_global_cluster_global_cluster_name.
- **create_rds_global_cluster_replica_cluster_region** - Region of the replica (secondary) cluster. **required**
- **create_rds_global_cluster_replica_cluster_instance_name** - Name of the instance in secondary cluster. **required**
- **create_rds_global_cluster_replica_enable_global_write_forwarding** - Whether to enable replica cluster to forward write operations to the primary cluster of an Amazon Aurora global database. Default is False. Supported only while creating new cluster. Choices include 'true', 'false, 'yes', 'no'.
- **create_rds_global_cluster_replica_cluster_vpc_security_group_ids** - A list of EC2 VPC security groups to associate with the replica DB cluster.

- **create_rds_global_cluster_operation** - Choices include 'create' and 'delete' to create or delete the resources.

Dependencies
------------

- role: [aws_setup_credentials](../aws_setup_credentials/README.md)

Example Playbook
----------------
```
---
- name: Playbook for demonstrating use of cloud.aws_ops.create_rds_global_cluster role
hosts: localhost
gather_facts: false
tasks:
- name: Create global db, primary cluster with instance & replica cluster with instance
ansible.builtin.include_role:
name: cloud.aws_ops.create_rds_global_cluster
vars:
create_rds_global_cluster_operation: create
create_rds_global_cluster_engine: aurora-mysql
create_rds_global_cluster_engine_version: 5.7
create_rds_global_cluster_instance_class: db.r5.large
create_rds_global_cluster_master_username: testusername
create_rds_global_cluster_master_user_password: test-password_rds
create_rds_global_cluster_global_cluster_name: test-cluster-global
create_rds_global_cluster_primary_cluster_name: test-cluster-primary
create_rds_global_cluster_primary_cluster_region: eu-central-1
create_rds_global_cluster_primary_cluster_instance_name: test-instance-primary
create_rds_global_cluster_replica_cluster_name: test-cluster-replica
create_rds_global_cluster_replica_cluster_region: us-west-2
create_rds_global_cluster_replica_enable_global_write_forwarding: true
create_rds_global_cluster_replica_cluster_instance_name: test-instance-replica
create_rds_global_cluster_primary_cluster_db_name: MyPrimaryDb
create_rds_global_cluster_primary_cluster_vpc_security_group_ids: [ "sg-03bfd123456789012", "sg-03bfd123456789034"]
create_rds_global_cluster_replica_cluster_vpc_security_group_ids: ["sg-03bfd123456789055"]

- name: Delete global db, primary cluster with instance & replica cluster with instance
ansible.builtin.include_role:
name: cloud.aws_ops.create_rds_global_cluster
vars:
create_rds_global_cluster_operation: delete
create_rds_global_cluster_global_cluster_name: test-cluster-global
create_rds_global_cluster_primary_cluster_name: test-cluster-primary
create_rds_global_cluster_primary_cluster_region: eu-central-1
create_rds_global_cluster_primary_cluster_instance_name: test-instance-primary
create_rds_global_cluster_replica_cluster_name: test-cluster-replica
create_rds_global_cluster_replica_cluster_region: us-west-2
create_rds_global_cluster_replica_cluster_instance_name: test-instance-replica
```

mandar242 marked this conversation as resolved.
Show resolved Hide resolved
License
-------
GNU General Public License v3.0 or later
8 changes: 8 additions & 0 deletions roles/create_rds_global_cluster/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
# defaults file for roles/create_rds_global_cluster
create_rds_global_cluster_engine: "aurora-postgresql"
create_rds_global_cluster_engine_version: 5.7
create_rds_global_cluster_primary_cluster_name: "{{ create_rds_global_cluster_global_cluster_name }}-primary"
create_rds_global_cluster_replica_cluster_name: "{{ create_rds_global_cluster_global_cluster_name }}-replica"
create_rds_global_cluster_primary_cluster_instance_name: "{{ create_rds_global_cluster_global_cluster_name }}-primary-instance"
create_rds_global_cluster_replica_cluster_instance_name: "{{ create_rds_global_cluster_global_cluster_name }}-replica-instance"
3 changes: 3 additions & 0 deletions roles/create_rds_global_cluster/meta/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
---
dependencies:
- role: cloud.aws_ops.aws_setup_credentials
54 changes: 54 additions & 0 deletions roles/create_rds_global_cluster/tasks/create.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
- name: Run 'create_rds_global_cluster' role create operations
module_defaults:
group/aws: "{{ aws_setup_credentials__output }}"
mandar242 marked this conversation as resolved.
Show resolved Hide resolved
group/amazon.cloud.aws: "{{ aws_setup_credentials__output }}"

block:
- name: Create rds global database
amazon.cloud.rds_global_cluster:
global_cluster_identifier: "{{ create_rds_global_cluster_global_cluster_name }}"
engine: "{{ create_rds_global_cluster_engine }}"
engine_version: "{{ create_rds_global_cluster_engine_version }}"
hakbailey marked this conversation as resolved.
Show resolved Hide resolved
aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}"
state: present

- name: Create a primary cluster for global database in "{{ create_rds_global_cluster_primary_cluster_region }}"
amazon.aws.rds_cluster:
db_cluster_identifier: "{{ create_rds_global_cluster_primary_cluster_name }}"
aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}"
engine: "{{ create_rds_global_cluster_engine }}"
engine_version: "{{ create_rds_global_cluster_engine_version }}"
master_username: "{{ create_rds_global_cluster_master_username }}"
master_user_password: "{{ create_rds_global_cluster_master_user_password }}"
db_subnet_group_name: "{{ create_rds_global_cluster_db_subnet_group_name | default(omit) }}"
global_cluster_identifier: "{{ create_rds_global_cluster_global_cluster_name }}"
database_name: "{{ create_rds_global_cluster_primary_cluster_db_name | default(omit) }}"
vpc_security_group_ids: "{{ create_rds_global_cluster_primary_cluster_vpc_security_group_ids | default(omit) }}"

- name: Create an instance connected to primary cluster
amazon.aws.rds_instance:
db_cluster_identifier: "{{ create_rds_global_cluster_primary_cluster_name }}"
hakbailey marked this conversation as resolved.
Show resolved Hide resolved
db_instance_identifier: "{{ create_rds_global_cluster_primary_cluster_instance_name }}"
aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}"
engine: "{{ create_rds_global_cluster_engine }}"
db_instance_class: "{{ create_rds_global_cluster_instance_class }}"

- name: Create a read replica cluster for global database in "{{ create_rds_global_cluster_replica_cluster_region }}"
amazon.aws.rds_cluster:
db_cluster_identifier: "{{ create_rds_global_cluster_replica_cluster_name }}"
aws_region: "{{ create_rds_global_cluster_replica_cluster_region }}"
engine: "{{ create_rds_global_cluster_engine }}"
engine_version: "{{ create_rds_global_cluster_engine_version }}"
db_subnet_group_name: "{{ create_rds_global_cluster_db_subnet_group_name | default(omit) }}"
global_cluster_identifier: "{{ create_rds_global_cluster_global_cluster_name }}"
enable_global_write_forwarding: "{{ create_rds_global_cluster_replica_enable_global_write_forwarding | default(omit) }}"
vpc_security_group_ids: "{{ create_rds_global_cluster_replica_cluster_vpc_security_group_ids | default(omit) }}"

- name: Create an instance connected to secondary cluster
amazon.aws.rds_instance:
db_cluster_identifier: "{{ create_rds_global_cluster_replica_cluster_name }}"
db_instance_identifier: "{{ create_rds_global_cluster_replica_cluster_instance_name }}"
aws_region: "{{ create_rds_global_cluster_replica_cluster_region }}"
engine: "{{ create_rds_global_cluster_engine }}"
db_instance_class: "{{ create_rds_global_cluster_instance_class }}"
77 changes: 77 additions & 0 deletions roles/create_rds_global_cluster/tasks/delete.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
---
- name: Run 'create_rds_global_cluster' role delete operations
module_defaults:
group/aws: "{{ aws_setup_credentials__output }}"
group/amazon.cloud.aws: "{{ aws_setup_credentials__output }}"

block:
- name: Get replica cluster info
amazon.aws.rds_cluster_info:
cluster_id: "{{ create_rds_global_cluster_replica_cluster_name }}"
aws_region: "{{ create_rds_global_cluster_replica_cluster_region }}"
register: create_rds_global_cluster_replica_cluster_info

- name: If replica cluster exists
when: create_rds_global_cluster_replica_cluster_info.clusters | length != 0
block:
- name: Delete instance attached to replica cluster if present
amazon.aws.rds_instance:
db_instance_identifier: "{{ create_rds_global_cluster_replica_cluster_instance_name }}"
aws_region: "{{ create_rds_global_cluster_replica_cluster_region }}"
skip_final_snapshot: true
wait: false
state: absent

- name: Delete replica cluster without creating a final snapshot
amazon.aws.rds_cluster:
cluster_id: "{{ create_rds_global_cluster_replica_cluster_name }}"
aws_region: "{{ create_rds_global_cluster_replica_cluster_region }}"
global_cluster_identifier: "{{ create_rds_global_cluster_global_cluster_name }}"
remove_from_global_db: true
skip_final_snapshot: true
state: absent

- name: Get primary cluster info
amazon.aws.rds_cluster_info:
cluster_id: "{{ create_rds_global_cluster_primary_cluster_name }}"
aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}"
register: create_rds_global_cluster_primary_cluster_info

- name: If primary cluster exists
when: create_rds_global_cluster_primary_cluster_info.clusters | length != 0
block:
- name: Delete instance attached to primary cluster if present
amazon.aws.rds_instance:
db_instance_identifier: "{{ create_rds_global_cluster_primary_cluster_instance_name }}"
aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}"
skip_final_snapshot: true
wait: false
state: absent

- name: Delete primary cluster without creating a final snapshot
amazon.aws.rds_cluster:
cluster_id: "{{ create_rds_global_cluster_primary_cluster_name }}"
aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}"
global_cluster_identifier: "{{ create_rds_global_cluster_global_cluster_name }}"
skip_final_snapshot: true
state: absent


- name: Delete the global cluster
amazon.cloud.rds_global_cluster:
global_cluster_identifier: "{{ create_rds_global_cluster_global_cluster_name }}"
aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}"
state: absent
register: create_rds_global_cluster_global_cluster_delete

- name: Print success
ansible.builtin.debug:
msg:
- 'Global cluster {{ create_rds_global_cluster_global_cluster_name }} deleted successfully'
when: create_rds_global_cluster_global_cluster_delete is not failed

- name: Print failure
ansible.builtin.debug:
msg:
- 'Global cluster {{ create_rds_global_cluster_global_cluster_name }} deletion failed'
when: create_rds_global_cluster_global_cluster_delete is failed
5 changes: 5 additions & 0 deletions roles/create_rds_global_cluster/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
- name: Run 'create_rds_global_cluster' role
block:
- name: Include file
ansible.builtin.include_tasks: "{{ create_rds_global_cluster_operation }}.yml"
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# reason: missing support for multi-region tests
unsupported

cloud/aws
role/create_rds_global_cluster
time=10m
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
test_engine: aurora-mysql
test_engine_version: 5.7
test_instance_class: db.r5.large
test_username: testrdsusername
test_password: test-rds_password
test_primary_cluster_subnet_group_name: ''

# Global cluster parameters ================================
test_global_cluster_name: ansible-test-global-{{ tiny_prefix }}

# Primary cluster parameters ================================
test_primary_cluster_name: ansible-test-primary-{{ tiny_prefix }}
test_primary_cluster_region: eu-central-1
test_primary_cluster_instance_name: ansible-test-instance-primary-{{ tiny_prefix }}

# Replica cluster parameters ================================
test_replica_cluster_name: ansible-test-replica-{{ tiny_prefix }}
test_replica_cluster_region: us-west-2
test_replica_enable_global_write_forwarding: true
test_replica_cluster_instance_name: ansible-test-instance-replica-{{ tiny_prefix }}

aws_security_token: '{{ security_token | default(omit) }}'
Loading