diff --git a/README.md b/README.md index f8b624d4..b8e69f79 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,7 @@ Name | Description [cloud.aws_ops.awsconfig_apigateway_with_lambda_integration](https://github.com/ansible-collections/cloud.aws_ops/blob/main/roles/awsconfig_apigateway_with_lambda_integration/README.md)|A role to create/delete an API gateway with lambda function integration. [cloud.aws_ops.manage_transit_gateway](https://github.com/ansible-collections/cloud.aws_ops/blob/main/roles/manage_transit_gateway/README.md)|A role to create/delete transit_gateway with vpc and vpn attachments. [cloud.aws_ops.deploy_flask_app](https://github.com/ansible-collections/cloud.aws_ops/blob/main/roles/deploy_flask_app/README.md)|A role to deploy a flask web application on AWS. +[cloud.aws_ops.create_rds_global_cluster](https://github.com/ansible-collections/cloud.aws_ops/blob/main/roles/create_rds_global_cluster/README.md)|A role to create, delete aurora global cluster with a primary cluster and a replica cluster in different regions. ### Playbooks Name | Description diff --git a/changelogs/fragments/create_rds_global_cluster_role.yml b/changelogs/fragments/create_rds_global_cluster_role.yml new file mode 100644 index 00000000..bb33813b --- /dev/null +++ b/changelogs/fragments/create_rds_global_cluster_role.yml @@ -0,0 +1,3 @@ +--- +minor_changes: + - create_rds_global_cluster - new role to create aurora global cluster with a primary and a replica cluster in different regions. diff --git a/galaxy.yml b/galaxy.yml index d74fe5af..509ff62e 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -19,6 +19,7 @@ tags: dependencies: amazon.aws: '>=5.1.0' community.aws: '>=5.0.0' + amazon.cloud: '>=0.4.0' version: 1.0.3 build_ignore: - .DS_Store diff --git a/roles/create_rds_global_cluster/README.md b/roles/create_rds_global_cluster/README.md new file mode 100644 index 00000000..51f8202b --- /dev/null +++ b/roles/create_rds_global_cluster/README.md @@ -0,0 +1,107 @@ +create_rds_global_cluster +========= + +A role to create an Amazon Aurora global cluster with two different region rds clusters. + +Creates the following resources: +1. Global Cluster - Amazon Aurora Postgresql or Amazon Aurora MySql cluster. If `create_rds_global_cluster_engine` is not provided, defaults to Amazon Aurora Postgresql. +2. Primary Cluster - Primary cluster in specified region (`create_rds_global_cluster_primary_cluster_region`). +3. Primary Cluster Instance - Instance in the primary cluster. +4. Replica (secondary) Cluster - Secondary cluster in specified region (`create_rds_global_cluster_replica_cluster_region`). +5. Replica Cluster Instance - Instance in the replica cluster. + +Please refer to [Role Variables](#role-variables) for variables and usage. + +Requirements +------------ + +AWS User Account with the following permissions: + +* rds:CreateGlobalCluster +* rds:DeleteGlobalCluster +* rds:ModifyGlobalCluster +* rds:CreateDBCluster +* rds:DeleteDBCluster +* rds:ModifyDBCluster +* rds:DescribeGlobalClusters +* rds:DescribeDBClusters + +Role Variables +-------------- +**Global cluster variables** +- **create_rds_global_cluster_global_cluster_name** - Name of the Amazon Aurora global cluster. **required** +- **create_rds_global_cluster_engine** - Engine of the Amazon Aurora global and rds clusters. Default is aurora-postgresql. +- **create_rds_global_cluster_engine_version** - Engine version of the Amazon Aurora global and rds clusters. +- **create_rds_global_cluster_instance_class** - Instance class of instance in primary and replica cluster. **required** +- **create_rds_global_cluster_master_username** - Username of the rds clusters master user. **required** +- **create_rds_global_cluster_master_user_password** - Password of the rds clusters master user. **required** + +**Primary cluster variables** +- **create_rds_global_cluster_primary_cluster_name** - Name of the primary cluster. Default is $create_rds_global_cluster_global_cluster_name. +- **create_rds_global_cluster_primary_cluster_region** - Region of the primary cluster. **required** +- **create_rds_global_cluster_primary_cluster_instance_name** - Name of the instance in primary cluster. **required** +- **create_rds_global_cluster_primary_cluster_db_name** - The name for your database of up to 64 alphanumeric characters. If not provided, database is not created in the cluster. +- **create_rds_global_cluster_primary_cluster_vpc_security_group_ids** - A list of EC2 VPC security groups to associate with the primary DB cluster. +- **create_rds_global_cluster_db_subnet_group_name** - A DB subnet group to associate with this DB cluster if not using the default. + +**Replica cluster variables** +- **create_rds_global_cluster_replica_cluster_name** - Name of the replica (secondary) cluster. Default is create_rds_global_cluster_global_cluster_name. +- **create_rds_global_cluster_replica_cluster_region** - Region of the replica (secondary) cluster. **required** +- **create_rds_global_cluster_replica_cluster_instance_name** - Name of the instance in secondary cluster. **required** +- **create_rds_global_cluster_replica_enable_global_write_forwarding** - Whether to enable replica cluster to forward write operations to the primary cluster of an Amazon Aurora global database. Default is False. Supported only while creating new cluster. Choices include 'true', 'false, 'yes', 'no'. +- **create_rds_global_cluster_replica_cluster_vpc_security_group_ids** - A list of EC2 VPC security groups to associate with the replica DB cluster. + +- **create_rds_global_cluster_operation** - Choices include 'create' and 'delete' to create or delete the resources. + +Dependencies +------------ + +- role: [aws_setup_credentials](../aws_setup_credentials/README.md) + +Example Playbook +---------------- +``` +--- +- name: Playbook for demonstrating use of cloud.aws_ops.create_rds_global_cluster role + hosts: localhost + gather_facts: false + tasks: + - name: Create global db, primary cluster with instance & replica cluster with instance + ansible.builtin.include_role: + name: cloud.aws_ops.create_rds_global_cluster + vars: + create_rds_global_cluster_operation: create + create_rds_global_cluster_engine: aurora-mysql + create_rds_global_cluster_engine_version: 5.7 + create_rds_global_cluster_instance_class: db.r5.large + create_rds_global_cluster_master_username: testusername + create_rds_global_cluster_master_user_password: test-password_rds + create_rds_global_cluster_global_cluster_name: test-cluster-global + create_rds_global_cluster_primary_cluster_name: test-cluster-primary + create_rds_global_cluster_primary_cluster_region: eu-central-1 + create_rds_global_cluster_primary_cluster_instance_name: test-instance-primary + create_rds_global_cluster_replica_cluster_name: test-cluster-replica + create_rds_global_cluster_replica_cluster_region: us-west-2 + create_rds_global_cluster_replica_enable_global_write_forwarding: true + create_rds_global_cluster_replica_cluster_instance_name: test-instance-replica + create_rds_global_cluster_primary_cluster_db_name: MyPrimaryDb + create_rds_global_cluster_primary_cluster_vpc_security_group_ids: [ "sg-03bfd123456789012", "sg-03bfd123456789034"] + create_rds_global_cluster_replica_cluster_vpc_security_group_ids: ["sg-03bfd123456789055"] + + - name: Delete global db, primary cluster with instance & replica cluster with instance + ansible.builtin.include_role: + name: cloud.aws_ops.create_rds_global_cluster + vars: + create_rds_global_cluster_operation: delete + create_rds_global_cluster_global_cluster_name: test-cluster-global + create_rds_global_cluster_primary_cluster_name: test-cluster-primary + create_rds_global_cluster_primary_cluster_region: eu-central-1 + create_rds_global_cluster_primary_cluster_instance_name: test-instance-primary + create_rds_global_cluster_replica_cluster_name: test-cluster-replica + create_rds_global_cluster_replica_cluster_region: us-west-2 + create_rds_global_cluster_replica_cluster_instance_name: test-instance-replica +``` + +License +------- +GNU General Public License v3.0 or later diff --git a/roles/create_rds_global_cluster/defaults/main.yml b/roles/create_rds_global_cluster/defaults/main.yml new file mode 100644 index 00000000..d4f18f2c --- /dev/null +++ b/roles/create_rds_global_cluster/defaults/main.yml @@ -0,0 +1,8 @@ +--- +# defaults file for roles/create_rds_global_cluster +create_rds_global_cluster_engine: "aurora-postgresql" +create_rds_global_cluster_engine_version: 5.7 +create_rds_global_cluster_primary_cluster_name: "{{ create_rds_global_cluster_global_cluster_name }}-primary" +create_rds_global_cluster_replica_cluster_name: "{{ create_rds_global_cluster_global_cluster_name }}-replica" +create_rds_global_cluster_primary_cluster_instance_name: "{{ create_rds_global_cluster_global_cluster_name }}-primary-instance" +create_rds_global_cluster_replica_cluster_instance_name: "{{ create_rds_global_cluster_global_cluster_name }}-replica-instance" diff --git a/roles/create_rds_global_cluster/meta/main.yml b/roles/create_rds_global_cluster/meta/main.yml new file mode 100644 index 00000000..e8b3ab42 --- /dev/null +++ b/roles/create_rds_global_cluster/meta/main.yml @@ -0,0 +1,3 @@ +--- +dependencies: + - role: cloud.aws_ops.aws_setup_credentials diff --git a/roles/create_rds_global_cluster/tasks/create.yml b/roles/create_rds_global_cluster/tasks/create.yml new file mode 100644 index 00000000..d7f4a171 --- /dev/null +++ b/roles/create_rds_global_cluster/tasks/create.yml @@ -0,0 +1,54 @@ +--- +- name: Run 'create_rds_global_cluster' role create operations + module_defaults: + group/aws: "{{ aws_setup_credentials__output }}" + group/amazon.cloud.aws: "{{ aws_setup_credentials__output }}" + + block: + - name: Create rds global database + amazon.cloud.rds_global_cluster: + global_cluster_identifier: "{{ create_rds_global_cluster_global_cluster_name }}" + engine: "{{ create_rds_global_cluster_engine }}" + engine_version: "{{ create_rds_global_cluster_engine_version }}" + aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}" + state: present + + - name: Create a primary cluster for global database in "{{ create_rds_global_cluster_primary_cluster_region }}" + amazon.aws.rds_cluster: + db_cluster_identifier: "{{ create_rds_global_cluster_primary_cluster_name }}" + aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}" + engine: "{{ create_rds_global_cluster_engine }}" + engine_version: "{{ create_rds_global_cluster_engine_version }}" + master_username: "{{ create_rds_global_cluster_master_username }}" + master_user_password: "{{ create_rds_global_cluster_master_user_password }}" + db_subnet_group_name: "{{ create_rds_global_cluster_db_subnet_group_name | default(omit) }}" + global_cluster_identifier: "{{ create_rds_global_cluster_global_cluster_name }}" + database_name: "{{ create_rds_global_cluster_primary_cluster_db_name | default(omit) }}" + vpc_security_group_ids: "{{ create_rds_global_cluster_primary_cluster_vpc_security_group_ids | default(omit) }}" + + - name: Create an instance connected to primary cluster + amazon.aws.rds_instance: + db_cluster_identifier: "{{ create_rds_global_cluster_primary_cluster_name }}" + db_instance_identifier: "{{ create_rds_global_cluster_primary_cluster_instance_name }}" + aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}" + engine: "{{ create_rds_global_cluster_engine }}" + db_instance_class: "{{ create_rds_global_cluster_instance_class }}" + + - name: Create a read replica cluster for global database in "{{ create_rds_global_cluster_replica_cluster_region }}" + amazon.aws.rds_cluster: + db_cluster_identifier: "{{ create_rds_global_cluster_replica_cluster_name }}" + aws_region: "{{ create_rds_global_cluster_replica_cluster_region }}" + engine: "{{ create_rds_global_cluster_engine }}" + engine_version: "{{ create_rds_global_cluster_engine_version }}" + db_subnet_group_name: "{{ create_rds_global_cluster_db_subnet_group_name | default(omit) }}" + global_cluster_identifier: "{{ create_rds_global_cluster_global_cluster_name }}" + enable_global_write_forwarding: "{{ create_rds_global_cluster_replica_enable_global_write_forwarding | default(omit) }}" + vpc_security_group_ids: "{{ create_rds_global_cluster_replica_cluster_vpc_security_group_ids | default(omit) }}" + + - name: Create an instance connected to secondary cluster + amazon.aws.rds_instance: + db_cluster_identifier: "{{ create_rds_global_cluster_replica_cluster_name }}" + db_instance_identifier: "{{ create_rds_global_cluster_replica_cluster_instance_name }}" + aws_region: "{{ create_rds_global_cluster_replica_cluster_region }}" + engine: "{{ create_rds_global_cluster_engine }}" + db_instance_class: "{{ create_rds_global_cluster_instance_class }}" diff --git a/roles/create_rds_global_cluster/tasks/delete.yml b/roles/create_rds_global_cluster/tasks/delete.yml new file mode 100644 index 00000000..fd7e366f --- /dev/null +++ b/roles/create_rds_global_cluster/tasks/delete.yml @@ -0,0 +1,77 @@ +--- +- name: Run 'create_rds_global_cluster' role delete operations + module_defaults: + group/aws: "{{ aws_setup_credentials__output }}" + group/amazon.cloud.aws: "{{ aws_setup_credentials__output }}" + + block: + - name: Get replica cluster info + amazon.aws.rds_cluster_info: + cluster_id: "{{ create_rds_global_cluster_replica_cluster_name }}" + aws_region: "{{ create_rds_global_cluster_replica_cluster_region }}" + register: create_rds_global_cluster_replica_cluster_info + + - name: If replica cluster exists + when: create_rds_global_cluster_replica_cluster_info.clusters | length != 0 + block: + - name: Delete instance attached to replica cluster if present + amazon.aws.rds_instance: + db_instance_identifier: "{{ create_rds_global_cluster_replica_cluster_instance_name }}" + aws_region: "{{ create_rds_global_cluster_replica_cluster_region }}" + skip_final_snapshot: true + wait: false + state: absent + + - name: Delete replica cluster without creating a final snapshot + amazon.aws.rds_cluster: + cluster_id: "{{ create_rds_global_cluster_replica_cluster_name }}" + aws_region: "{{ create_rds_global_cluster_replica_cluster_region }}" + global_cluster_identifier: "{{ create_rds_global_cluster_global_cluster_name }}" + remove_from_global_db: true + skip_final_snapshot: true + state: absent + + - name: Get primary cluster info + amazon.aws.rds_cluster_info: + cluster_id: "{{ create_rds_global_cluster_primary_cluster_name }}" + aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}" + register: create_rds_global_cluster_primary_cluster_info + + - name: If primary cluster exists + when: create_rds_global_cluster_primary_cluster_info.clusters | length != 0 + block: + - name: Delete instance attached to primary cluster if present + amazon.aws.rds_instance: + db_instance_identifier: "{{ create_rds_global_cluster_primary_cluster_instance_name }}" + aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}" + skip_final_snapshot: true + wait: false + state: absent + + - name: Delete primary cluster without creating a final snapshot + amazon.aws.rds_cluster: + cluster_id: "{{ create_rds_global_cluster_primary_cluster_name }}" + aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}" + global_cluster_identifier: "{{ create_rds_global_cluster_global_cluster_name }}" + skip_final_snapshot: true + state: absent + + + - name: Delete the global cluster + amazon.cloud.rds_global_cluster: + global_cluster_identifier: "{{ create_rds_global_cluster_global_cluster_name }}" + aws_region: "{{ create_rds_global_cluster_primary_cluster_region }}" + state: absent + register: create_rds_global_cluster_global_cluster_delete + + - name: Print success + ansible.builtin.debug: + msg: + - 'Global cluster {{ create_rds_global_cluster_global_cluster_name }} deleted successfully' + when: create_rds_global_cluster_global_cluster_delete is not failed + + - name: Print failure + ansible.builtin.debug: + msg: + - 'Global cluster {{ create_rds_global_cluster_global_cluster_name }} deletion failed' + when: create_rds_global_cluster_global_cluster_delete is failed diff --git a/roles/create_rds_global_cluster/tasks/main.yml b/roles/create_rds_global_cluster/tasks/main.yml new file mode 100644 index 00000000..32970f27 --- /dev/null +++ b/roles/create_rds_global_cluster/tasks/main.yml @@ -0,0 +1,5 @@ +--- +- name: Run 'create_rds_global_cluster' role + block: + - name: Include file + ansible.builtin.include_tasks: "{{ create_rds_global_cluster_operation }}.yml" diff --git a/tests/integration/targets/test_create_rds_global_cluster/aliases b/tests/integration/targets/test_create_rds_global_cluster/aliases new file mode 100644 index 00000000..387cc350 --- /dev/null +++ b/tests/integration/targets/test_create_rds_global_cluster/aliases @@ -0,0 +1,6 @@ +# reason: missing support for multi-region tests +unsupported + +cloud/aws +role/create_rds_global_cluster +time=10m diff --git a/tests/integration/targets/test_create_rds_global_cluster/defaults/main.yml b/tests/integration/targets/test_create_rds_global_cluster/defaults/main.yml new file mode 100644 index 00000000..c09a2482 --- /dev/null +++ b/tests/integration/targets/test_create_rds_global_cluster/defaults/main.yml @@ -0,0 +1,23 @@ +--- +test_engine: aurora-mysql +test_engine_version: 5.7 +test_instance_class: db.r5.large +test_username: testrdsusername +test_password: test-rds_password +test_primary_cluster_subnet_group_name: '' + +# Global cluster parameters ================================ +test_global_cluster_name: ansible-test-global-{{ tiny_prefix }} + +# Primary cluster parameters ================================ +test_primary_cluster_name: ansible-test-primary-{{ tiny_prefix }} +test_primary_cluster_region: eu-central-1 +test_primary_cluster_instance_name: ansible-test-instance-primary-{{ tiny_prefix }} + +# Replica cluster parameters ================================ +test_replica_cluster_name: ansible-test-replica-{{ tiny_prefix }} +test_replica_cluster_region: us-west-2 +test_replica_enable_global_write_forwarding: true +test_replica_cluster_instance_name: ansible-test-instance-replica-{{ tiny_prefix }} + +aws_security_token: '{{ security_token | default(omit) }}' diff --git a/tests/integration/targets/test_create_rds_global_cluster/tasks/main.yml b/tests/integration/targets/test_create_rds_global_cluster/tasks/main.yml new file mode 100644 index 00000000..9a52a646 --- /dev/null +++ b/tests/integration/targets/test_create_rds_global_cluster/tasks/main.yml @@ -0,0 +1,81 @@ +--- +- name: Integration tests for 'create_rds_global_cluster' role + module_defaults: + group/amazon.cloud.aws: + aws_access_key: "{{ aws_access_key }}" + aws_secret_key: "{{ aws_secret_key }}" + security_token: "{{ security_token | default(omit) }}" + region: "{{ aws_region }}" + group/aws: + aws_access_key: "{{ aws_access_key }}" + aws_secret_key: "{{ aws_secret_key }}" + security_token: "{{ security_token | default(omit) }}" + region: "{{ aws_region }}" + + # Due to CI limitations, it is not possible to create a replica in another region + # and with AWS limitations we cannot create a replica cluster in same region as primary cluster i.e. us-east-1 + # can be run on team account + block: + - name: Create global db + ansible.builtin.include_role: + name: cloud.aws_ops.create_rds_global_cluster + vars: + create_rds_global_cluster_operation: create + create_rds_global_cluster_engine: "{{ test_engine }}" + create_rds_global_cluster_engine_version: "{{ test_engine_version }}" + create_rds_global_cluster_instance_class: "{{ test_instance_class }}" + create_rds_global_cluster_master_username: "{{ test_username }}" + create_rds_global_cluster_master_user_password: "{{ test_password }}" + create_rds_global_cluster_global_cluster_name: "{{ test_global_cluster_name }}" + create_rds_global_cluster_primary_cluster_name: "{{ test_primary_cluster_name }}" + create_rds_global_cluster_primary_cluster_region: "{{ test_primary_cluster_region }}" + create_rds_global_cluster_primary_cluster_instance_name: "{{ test_primary_cluster_instance_name }}" + create_rds_global_cluster_replica_cluster_name: "{{ test_replica_cluster_name }}" + create_rds_global_cluster_replica_cluster_region: "{{ test_replica_cluster_region }}" + create_rds_global_cluster_replica_cluster_instance_name: "{{ test_replica_cluster_instance_name }}" + create_rds_global_cluster_replica_enable_global_write_forwarding: "{{ test_replica_enable_global_write_forwarding }}" + + - name: Get Global DB information + amazon.cloud.rds_global_cluster: + global_cluster_identifier: "{{ test_global_cluster_name }}" + region: "{{ test_primary_cluster_region }}" + state: describe + register: global_cluster_info + + - name: Get primary DB cluster information + amazon.aws.rds_cluster_info: + cluster_id: "{{ test_primary_cluster_name }}" + region: "{{ test_primary_cluster_region }}" + register: primary_cluster_info + + - name: Get secondary DB cluster information + amazon.aws.rds_cluster_info: + cluster_id: "{{ test_replica_cluster_name }}" + region: "{{ test_replica_cluster_region }}" + register: replica_cluster_info + + - name: Verify resources are created successfully + ansible.builtin.assert: + that: + - global_cluster_info.result.identifier == "{{ test_global_cluster_name }}" + - global_cluster_info.result.properties.source_db_cluster_identifier == primary_cluster_info.clusters[0].db_cluster_arn + - primary_cluster_info.clusters[0].db_cluster_identifier == "{{ test_primary_cluster_name }}" + - "'{{ test_primary_cluster_region }}' in primary_cluster_info.clusters[0].availability_zones[0]" + - replica_cluster_info.clusters[0].db_cluster_identifier == "{{ test_replica_cluster_name }}" + - "'{{ test_replica_cluster_region }}' in replica_cluster_info.clusters[0].availability_zones[0]" + - replica_cluster_info.clusters[0].replication_source_identifier == primary_cluster_info.clusters[0].db_cluster_arn + + always: + + - name: Delete global db and other role infrastructure + ansible.builtin.include_role: + name: cloud.aws_ops.create_rds_global_cluster + vars: + create_rds_global_cluster_operation: delete + create_rds_global_cluster_global_cluster_name: "{{ test_global_cluster_name }}" + create_rds_global_cluster_primary_cluster_name: "{{ test_primary_cluster_name }}" + create_rds_global_cluster_primary_cluster_region: "{{ test_primary_cluster_region }}" + create_rds_global_cluster_primary_cluster_instance_name: "{{ test_primary_cluster_instance_name }}" + create_rds_global_cluster_replica_cluster_name: "{{ test_replica_cluster_name }}" + create_rds_global_cluster_replica_cluster_region: "{{ test_replica_cluster_region }}" + create_rds_global_cluster_replica_cluster_instance_name: "{{ test_replica_cluster_instance_name }}"