Merge pull request #1 from ZacharyWallace/main

Merging in initial templates for taskcat functional test debugging
aws-ia · May 15, 2023 · 8c1f5ed · 8c1f5ed
2 parents 7213f9f + 38db50d
commit 8c1f5ed
Showing 9 changed files with 1,012 additions and 33 deletions.
diff --git a/.gitmodules b/.gitmodules
@@ -0,0 +1,6 @@
+[submodule "submodules/cfn-abi-aws-cloudtrail"]
+	path = submodules/cfn-abi-aws-cloudtrail
+	url = git@github.com:aws-ia/cfn-abi-aws-cloudtrail.git
+[submodule "submodules/cfn-abi-amazon-guardduty"]
+	path = submodules/cfn-abi-amazon-guardduty
+	url = git@github.com:aws-ia/cfn-abi-amazon-guardduty.git
diff --git a/.taskcat.yml b/.taskcat.yml
@@ -1,33 +1,24 @@
 project:
-  name: update-me-to-project-repo-name
-  owner: quickstart@amazon.com
+  name: cfn-abi-deepwatch-mdr
+  owner: zachary.wallace@deepwatch.com
   package_lambda: false
+  shorten_stack_name: true
+  s3_regional_buckets: false
   regions:
-  - ap-northeast-1
-  - ap-northeast-2
-  - ap-southeast-1
-  - ap-southeast-2
-  - eu-central-1
-  - eu-west-1
-  - sa-east-1
   - us-east-1
-  - us-west-1
-  - us-west-2
 tests:
-  sample:
-    parameters:
-      Param1: 'Inputs to Stack'
-      # Examples: of other taskcat dynamic input parameters for more into see http://taskcat.io
-      #
-      #      AvailabilityZones: $[taskcat_genaz_3]
-      #      ByteValue: 1
-      #      PasswordA: $[taskcat_genpass_8A]
-      #      PasswordB: $[taskcat_genpass_32S]
-      #      RandomNumber: $[taskcat_random-numbers]
-      #      RandomString: $[taskcat_random-string]
-      #      StackName: TestStack
-      #      UUID: $[taskcat_genuuid]
-      #
+  logging-top-level:
     regions:
-    - us-east-1
-    template: templates/sample-workload.template.yaml
+      - us-east-1
+    template: templates/deepwatch-root-stack.yaml
+    parameters:
+      pSRAStagingS3KeyPrefix: $[taskcat_project_name]
+      pSRASourceS3BucketName: $[taskcat_autobucket]
+      pSRASourceS3BucketNamePrefix: $[taskcat_project_name]
+      pSRAS3BucketRegion: us-east-1
+      pAutoEnableS3Logs: 'true'
+      pAutoEnableK8sLogs: 'false'
+      pAutoEnableMalwareProtection: 'false'
+      pSRASolutionName: sra-guardduty-org
+      pSraTestingFlag: 'true'
+      pDeepwatchRoleName: 'deepwatch-mdr-role'
diff --git a/README.md b/README.md
@@ -0,0 +1,76 @@
+# Deepwatch Amazon Built In Module 
+
+## Overview
+
+<img width="742" height="751" src="images/overview-architecture.jpg" alt="Overview Architecture">
+
+This solution utilizes CloudFormation to deploy three solutions as one:
+
+- Solution A: A CloudFormation Nested Stack that deploys an Organization CloudTrail solution that will create an Organization CloudTrail within the Organization Management Account that is encrypted with a Customer Managed KMS Key managed in the Audit Account and logs delivered to the Log Archive Account. An Organization CloudTrail logs all events for all AWS accounts in the AWS Organization.
+- Solution B: A CloudFormation Nested Stack that enables GuardDuty for all existing AWS accounts in an AWS Organization and turns on the Auto-Enable feature for future accounts. The solution allows you to choose the regions in which to enable GuardDuty and delegates the GuardDuty administrator role to the organization's Audit account. It creates an S3 bucket in the logging account to collect aggregated findings from all accounts and assigns a lifecycle policy to transition data to Glacier storage after 365 days. The solution also enables GuardDuty S3 and EKS protection by default. 
+- Solution C: A StackSet in the logging account account where the previous solutions were configured to store logs to and sets ups all of the resources required to begin ingesting those logs to the Deepwatch Managed Detection & Response platform, including all necessary Lambdas, SNS Topics, SQS Queues, S3 Event Notifications, and IAM Roles & Policies. The outputs of this StackSet are all that is needed to finish setting up ingestion of your organizations CloudTrail and GuardDuty logs.
+
+## Getting Started
+
+To deploy this CloudFormation Stack via the AWS Console follow these steps:
+
+1. Clone this repository to your local machine.
+2. Open the AWS Management Console and navigate to CloudFormation.
+3. Create a new CloudFormation Stack.
+4. Choose to upload a template file and select the "deepwatch-root-stack.yaml" file.
+5. Enter your desired Stack name and fill in the required parameters.
+6. Click "Create Stack" to begin the deployment process.
+
+To deploy the CloudFormation stack using the AWS CLI follow these steps:
+
+1. Clone this repository to your local machine.
+2. Install and configure the AWS CLI on your machine.
+3. Open your terminal or command prompt and navigate to the directory where you cloned the repository.
+4. Run the following command to create the CloudFormation stack:
+
+   ```
+   aws cloudformation create-stack --stack-name <YOUR_STACK_NAME> --template-body file://deepwatch-root-stack.yaml --parameters ParameterKey=<PARAMETER_NAME>,ParameterValue=<PARAMETER_VALUE> --capabilities CAPABILITY_NAMED_IAM
+   ```
+
+   Be sure to replace `<YOUR_STACK_NAME>`, `<PARAMETER_NAME>`, and `<PARAMETER_VALUE>` with your desired values for the stack name and parameters.
+
+5. Wait for the stack to finish deploying. You can check the status of the deployment by running the following command:
+
+   ```
+   aws cloudformation describe-stacks --stack-name <YOUR_STACK_NAME>
+   ```
+
+   The stack status will be returned in the output.
+
+Once the stack has finished deploying, you can access the resources created by the stack via the AWS Management Console or the AWS CLI.
+
+## Deepwatch StackSet Resources 
+
+[diagram here]
+
+The Deepwatch CloudFormation StackSet creates several AWS resources:
+
+- `GuardDutyQueue`: an SQS queue that receives events for preprocessing.
+- `GuardDutyQueuePolicy`: an SQS queue policy that grants permissions to access the preprocessing queue.
+- `GuardDutyDeadLetterQueue`: an SQS queue that acts as a dead-letter queue for a preprocessing queue.
+- `GuardDutyDeadLetterQueuePolicy`: an SQS queue policy that grants permissions to access the dead-letter queue for the preprocessing queue.
+- `ControlTowerSNSTopic`: an SNS topic that sends notifications about events to the preprocessing queue.
+- `ControlTowerPreProcessedQueue`: an SQS queue that receives events for preprocessing.
+- `ControlTowerPreProcessedQueuePolicy`: an SQS queue policy that grants permissions to access the preprocessing queue.
+- `ControlTowerPreProcessedDeadLetterQueue`: an SQS queue that acts as a dead-letter queue for a preprocessing queue.
+- `ControlTowerPreProcessedDeadLetterQueuePolicy`: an SQS queue policy that grants permissions to access the dead-letter queue for the preprocessing queue.
+- `ControlTowerPreProcessedLambdaFunction`: a lambda function that is triggered by message arrival to the ControlTowerPreProcessedQueue to process CloudTrail logs.
+- `CloudTrailQueue`: an SQS queue that receives CloudTrail events.
+- `CloudTrailQueuePolicy`: an SQS queue policy that grants permissions to access the CloudTrail queue.
+- `CloudTrailDeadLetterQueue`: an SQS queue that acts as a dead-letter queue for the CloudTrailQueue.
+- `CloudTrailDeadLetterQueuePolicy`: an SQS queue policy that grants permissions to access the dead-letter queue for the CloudTrail queue.
+
+Additionally there is a custom resource that will place an event notification configuration on the GuardDuty and CloudTrail buckets to forward all new objectcreate events to the respective SQS queue/SNS Topic.
+
+## Post-Deployment Steps
+
+Following the deployment of the solution, please provide your Deepwatch engineer with the following outputs from the Deepwatch template:
+
+- `oCloudTrailQueueArn`
+- `oGuardDutyQueueArn`
+- `oDeepwatchRoleArn`
diff --git a/images/overview-architecture.jpg b/images/overview-architecture.jpg
diff --git a/scripts/cleanup_config.json b/scripts/cleanup_config.json
@@ -1,23 +1,60 @@
 [
+    {
+        "Type" : "S3_BUCKET",
+        "Filter" : "sra-guardduty-org-delivery-",
+        "Account" : "log_account"
+    },
+    {
+        "Type" : "S3_BUCKET",
+        "Filter" : "cfn-abi-amazon-guardduty-"
+    },
+    {
+        "Type" : "S3_BUCKET",
+        "Filter" : "sra-gd-staging-"
+    },
+    {
+        "Type" : "SSM_PARAMETER",
+        "Filter" : "/sra/gd/"
+    },
     {
         "Type": "STACK",
-        "Filter" : "replace-with-stack-name-prefix-to-be-cleaned"
+        "Filter" : "tcat-cfn-abi-deepwatch-mdr"
+    },
+    {
+        "Type": "STACK",
+        "Filter" : "tCaT-enable-cloudtrail-ssm-"
+    },
+    {
+        "Type" : "STACK",
+        "Filter": "tCaT-stage-ctrail-content-local-"
+    },
+    {
+        "Type" : "STACK",
+        "Filter": "org-level-trail-dataevent-only"
+    },
+    {
+        "Type" : "STACK",
+        "Filter": "org-level-trail-with-mgmt-events"
     },
     {
         "Type" : "S3_BUCKET",
-        "Filter": "replace-with-s3-bucket-prefix-to-be-cleaned"
+        "Filter": "sra-cloudtrail-staging-"
     },
     {
         "Type" : "S3_BUCKET",
-        "Filter": "replace-with-s3-bucket-prefix-to-be-cleaned",
-        "Account": "provide-your-account-name"
+        "Filter": "sra-org-trail-logs-",
+        "Account": "log_account"
+    },
+    {
+        "Type": "S3_BUCKET",
+        "Filter": "cfn-abi-aws-cloudtrail-"
     },
     {
         "Type" : "SSM_PARAMETER",
-        "Filter" : "/path/to/parameter-prefix-to-delete"
+        "Filter" : "/sra/ctrail/"
     },
     {
         "Type" : "LOG_GROUP",
-        "Filter" : "path/to/log-group-prefix-to-delete"
+        "Filter" : "sra/sra-org-trail"
     }
 ]
diff --git a/submodules/cfn-abi-amazon-guardduty b/submodules/cfn-abi-amazon-guardduty
diff --git a/submodules/cfn-abi-aws-cloudtrail b/submodules/cfn-abi-aws-cloudtrail
diff --git a/templates/deepwatch-logging-resource-configuration.yaml b/templates/deepwatch-logging-resource-configuration.yaml
diff --git a/templates/deepwatch-root-stack.yaml b/templates/deepwatch-root-stack.yaml
@@ -0,0 +1,168 @@
+AWSTemplateFormatVersion: 2010-09-09
+Description: >-
+  Configures services and resources for CloudTrail and GuardDuty logging in
+  Deepwatch cloud MDR
+Metadata:
+  'AWS::CloudFormation::Interface':
+    ParameterGroups:
+      - Label:
+          default: Source location details
+        Parameters:
+          - pSRAStagingS3KeyPrefix
+          - pSRASourceS3BucketName
+          - pSRAS3BucketRegion
+      - Label:
+          default: General Properties
+        Parameters:
+          - pSRASolutionName
+          - pSRAStagingS3KeyPrefix
+          - pSRAS3BucketRegion
+          - pDeepwatchRoleName
+      - Label:
+          default: Testing Properties
+        Parameters:
+          - pSraTestingFlag
+      - Label:
+          default: GuardDuty Configuration Properties
+        Parameters:
+          - pAutoEnableS3Logs
+          - pAutoEnableK8sLogs
+          - pAutoEnableMalwareProtection
+    ParameterLabels:
+      pSRAStagingS3KeyPrefix:
+        default: SRA Staging S3 Bucket Name Prefix
+      pSRASourceS3BucketName:
+        default: SRA Source S3 Location
+      pSRAS3BucketRegion:
+        default: SRA Bucket Region
+      pAutoEnableS3Logs:
+        default: Auto Enable S3 Logs
+      pAutoEnableK8sLogs:
+        default: Auto Enable kubernetes Logs
+      pAutoEnableMalwareProtection:
+        default: Auto Enable malware protection
+      pSRASolutionName:
+        default: SRA Solution Name
+      pSraTestingFlag:
+        default: Testing Flag for SRA
+      pDeepwatchRoleName:
+        default: IAM Role name for Deepwatch solution access
+Parameters:
+  pSRAStagingS3KeyPrefix:
+    AllowedValues:
+      - cfn-abi-deepwatch-mdr
+    Default: cfn-abi-deepwatch-mdr
+    Description: >-
+      SRA Staging S3 bucket name prefix for the SRA artifacts relevant to the
+      solutions. (e.g., lambda zips, CloudFormation templates). The account and
+      region are added to the prefix <bucket-name-prefix>-<account-id>-<region>.
+      Example = sra-staging-123456789012-us-east-1.
+    Type: String
+  pSRASourceS3BucketName:
+    AllowedPattern: '^([\w.-]{1,900})$|^(\/[\w.-]{1,900})*[\w.-]{1,900}$'
+    ConstraintDescription: >-
+      Must be alphanumeric or special characters [., _, -]. In addition, the
+      slash character ( / ) used to delineate hierarchies in parameter names.
+    Default: aws-abi-pilot
+    Description: >-
+      Source bucket for all templates and artefacts that will get copied into
+      staging bucket
+    Type: String
+  pSRAS3BucketRegion:
+    AllowedPattern: '^[a-z][a-z]-[a-z]*-[0-9]*$'
+    Type: String
+    Default: us-east-1
+  pAutoEnableS3Logs:
+    AllowedValues:
+      - 'true'
+      - 'false'
+    Default: 'true'
+    Description: Auto enable S3 logs
+    Type: String
+  pAutoEnableK8sLogs:
+    AllowedValues:
+      - 'true'
+      - 'false'
+    Default: 'false'
+    Description: Auto Enable kubernetes Logs
+    Type: String
+  pAutoEnableMalwareProtection:
+    AllowedValues:
+      - 'true'
+      - 'false'
+    Default: 'false'
+    Description: Auto Enable malware protection
+    Type: String
+  pSRASolutionName:
+    AllowedValues:
+      - sra-guardduty-org
+    Default: sra-guardduty-org
+    Description: >-
+      The SRA solution name. The default value is the folder name of the
+      solution
+    Type: String
+  pSraTestingFlag:
+    AllowedValues:
+      - 'true'
+      - 'false'
+    Default: 'false'
+    Type: String
+  pDeepwatchRoleName:
+    AllowedPattern: '^([\w.-]{1,900})$|^(\/[\w.-]{1,900})*[\w.-]{1,900}$'
+    ConstraintDescription: 'Must be alphanumeric or special characters [., _, -].'
+    Default: deepwatch-mdr-role
+    Description: The name of the role that will be created to provide access to ingest logs
+    Type: String
+Resources:
+  CloudTrailStack:
+    Type: 'AWS::CloudFormation::Stack'
+    Properties:
+      TemplateURL: !Sub >-
+        https://${pSRASourceS3BucketName}.s3.${pSRAS3BucketRegion}.${AWS::URLSuffix}/${pSRAStagingS3KeyPrefix}/submodules/cfn-abi-aws-cloudtrail/templates/sra-cloudtrail-enable-in-org-ssm.yaml
+      Parameters:
+        pSRAS3BucketRegion: !Ref pSRAS3BucketRegion
+        pEnableDataEventsOnly: false
+  GuardDutyStack:
+    Type: 'AWS::CloudFormation::Stack'
+    Properties:
+      TemplateURL: !Sub >-
+        https://${pSRASourceS3BucketName}.s3.${pSRAS3BucketRegion}.${AWS::URLSuffix}/${pSRAStagingS3KeyPrefix}/submodules/cfn-abi-amazon-guardduty/templates/sra-guardduty-enable-in-org-ssm.yaml
+      Parameters:
+        pAutoEnableS3Logs: !Ref pAutoEnableS3Logs
+        pAutoEnableK8sLogs: !Ref pAutoEnableK8sLogs
+        pAutoEnableMalwareProtection: !Ref pAutoEnableMalwareProtection
+        pSRASolutionName: !Ref pSRASolutionName
+        pSraTestingFlag: !Ref pSraTestingFlag
+        pSRAS3BucketRegion: !Ref pSRAS3BucketRegion        
+  DeepwatchResourceConfigurationStack:
+    Type: AWS::CloudFormation::StackSet
+    Properties:
+      StackSetName: deepwatch-logging-resource-configuration
+      AdministrationRoleARN: !Sub arn:${AWS::Partition}:iam::${AWS::AccountId}:role/service-role/AWSControlTowerStackSetRole
+      CallAs: SELF
+      Capabilities:
+        - CAPABILITY_NAMED_IAM
+      Description: Creates and configures all of the resources needed to begin ingestion of logs into Deepwatch MDR
+      ExecutionRoleName: AWSControlTowerExecution
+      ManagedExecution:
+        Active: true
+      OperationPreferences:
+        FailureTolerancePercentage: 0
+        MaxConcurrentPercentage: 100
+        RegionConcurrencyType: PARALLEL
+      PermissionModel: SELF_MANAGED
+      StackInstancesGroup:
+        - DeploymentTargets:
+            Accounts:
+              - !GetAtt [CloudTrailStack, Outputs.oLogArchiveAccountId]
+          Regions:
+            - !Ref AWS::Region
+      TemplateURL: !Sub >-
+        https://${pSRASourceS3BucketName}.s3.${pSRAS3BucketRegion}.${AWS::URLSuffix}/${pSRAStagingS3KeyPrefix}/templates/deepwatch-logging-resource-configuration.yaml
+      Parameters:
+        - ParameterKey: pGuardDutyBucketName
+          ParameterValue: !GetAtt [GuardDutyStack, Outputs.oPublishingDestinationBucketName]
+        - ParameterKey: pControlTowerBucketName
+          ParameterValue: !GetAtt [CloudTrailStack, Outputs.oOrganizationCloudTrailS3BucketName]
+        - ParameterKey: pDeepwatchRoleName
+          ParameterValue: !Ref pDeepwatchRoleName