Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: convert Lambda code from S3 binary object to ECR container image #626

Merged
merged 22 commits into from
May 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
7ab0279
refactor: convert Lambda code from S3 binary object to ECR container …
craigzour Mar 28, 2024
85ed505
refactor: update workflows to deploy new containerized Lambda functions
craigzour Apr 4, 2024
cf6d138
improved Localstack instance detection in local deployment script
craigzour Apr 19, 2024
3b91689
apply feedback from PR review
craigzour Apr 22, 2024
3859400
apply feedback from PR review
craigzour Apr 22, 2024
1760698
apply feedback from PR review
craigzour Apr 23, 2024
0cb4f16
apply feedback from PR review
craigzour Apr 23, 2024
8072b2c
apply feedback from PR review
craigzour Apr 23, 2024
3e83ae4
apply feedback from PR review
craigzour Apr 23, 2024
f04c0fb
Added Localstack readiness check to startup script
craigzour Apr 24, 2024
9c49664
Rearrange checks order
craigzour Apr 24, 2024
3ffe939
Fix Cloudwatch log groups name to align with lambda names
craigzour Apr 24, 2024
414f1db
chore: upgrade Hashicorp AWS from 5.31.0 to 5.32.0
craigzour Apr 30, 2024
2394044
Make sure we keep existing Cloudwatch log groups for Lambda functions…
craigzour Apr 30, 2024
bbaf5bf
put back old Terraform resource name to avoid deletion of Cloudwatch …
craigzour Apr 30, 2024
f297822
updated .lock.hcl files
craigzour Apr 30, 2024
533f1e7
fixed outputs
craigzour Apr 30, 2024
01e9a39
update one last .lock.hcl file
craigzour Apr 30, 2024
cba1c19
a few last fixes
craigzour Apr 30, 2024
d20d66b
found a way to keep the old Submission Lambda function name
craigzour May 1, 2024
3e734ae
fixed issue in localstack deploy lambda script
craigzour May 1, 2024
9ee1b9b
update .lock.hcl file for lambdas module
craigzour May 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/lambda-filter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
audit-logs: "lambda-code/audit-logs/**"
audit-logs-archiver: "lambda-code/audit-logs-archiver/**"
cognito-email-sender: "lambda-code/cognito-email-sender/**"
cognito-pre-sign-up: "lambda-code/cognito-pre-sign-up/**"
form-archiver: "lambda-code/form-archiver/**"
load-testing: "lambda-code/load-testing/**"
nagware: "lambda-code/nagware/**"
notify-slack: "lambda-code/notify-slack/**"
reliability: "lambda-code/reliability/**"
reliability-dlq-consumer: "lambda-code/reliability-dlq-consumer/**"
response-archiver: "lambda-code/response-archiver/**"
submission: "lambda-code/submission/**"
vault-integrity: "lambda-code/vault-integrity/**"
81 changes: 81 additions & 0 deletions .github/module-filter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
common: &common
craigzour marked this conversation as resolved.
Show resolved Hide resolved
- "env/common/**"
- "env/terragrunt.hcl"
- ".github/workflows/terragrunt-plan-staging.yml"
- ".github/workflows/terragrunt-apply-staging.yml"
alarms:
- *common
- "aws/alarms/**"
- "env/cloud/alarms/**"
app:
- *common
- "aws/app/**"
- "env/cloud/app/**"
cognito:
- *common
- "aws/cognito/**"
- "env/cloud/cognito/**"
dynamodb:
- *common
- "aws/dynamodb/**"
- "env/cloud/dynamodb/**"
ecr:
- *common
- "aws/ecr/**"
- "env/cloud/ecr/**"
file_scanning:
- *common
- "aws/file_scanning/**"
- "env/cloud/file_scanning/**"
hosted_zone:
- *common
- "aws/hosted_zone/**"
- "env/cloud/hosted_zone/**"
kms:
- *common
- "aws/kms/**"
- "env/cloud/kms/**"
lambdas:
- *common
- "aws/lambdas/**"
- "env/cloud/lambdas/**"
load_balancer:
- *common
- "aws/load_balancer/**"
- "env/cloud/load_balancer/**"
load_testing:
- *common
- "aws/load_testing/**"
- "env/cloud/load_testing/**"
network:
- *common
- "aws/network/**"
- "env/cloud/network/**"
oidc_roles:
- *common
- "aws/oidc_roles/**"
- "env/cloud/oidc_roles/**"
rds:
- *common
- "aws/rds/**"
- "env/cloud/rds/**"
redis:
- *common
- "aws/redis/**"
- "env/cloud/redis/**"
s3:
- *common
- "aws/s3/**"
- "env/cloud/s3/**"
secrets:
- *common
- "aws/secrets/**"
- "env/cloud/secrets/**"
sns:
- *common
- "aws/sns/**"
- "env/cloud/sns/**"
sqs:
- *common
- "aws/sqs/**"
- "env/cloud/sqs/**"
16 changes: 16 additions & 0 deletions .github/workflows/build-lambda-images/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Build Lambda images

inputs:
lambda-directory:
required: true
lambda-name:
required: true

runs:
using: "composite"
steps:
- run: docker build -t $LAMBDA_NAME-lambda .
env:
LAMBDA_NAME: ${{ inputs.lambda-name }}
working-directory: ${{ inputs.lambda-directory }}
shell: bash
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Request Lambda functions to use new image

inputs:
aws-role-to-assume:
required: true
aws-role-session-name:
required: true
aws-region:
required: true
lambda-name:
required: true
image-tag:
required: true

runs:
using: "composite"
steps:
- name: Configure AWS credentials using OIDC
uses: aws-actions/configure-aws-credentials@010d0da01d0b5a38af31e9c3470dbfdabdecca3a # v4.0.1
with:
role-to-assume: ${{ inputs.aws-role-to-assume }}
role-session-name: ${{ inputs.aws-role-session-name }}
aws-region: ${{ inputs.aws-region }}

- name: Login to Staging Amazon ECR
id: login-ecr-staging
uses: aws-actions/amazon-ecr-login@v2

- name: Update Lambda function image
env:
LAMBDA_NAME: ${{ inputs.lambda-name }}
IMAGE_TAG: ${{ inputs.image-tag }}
ECR_REGISTRY: ${{ steps.login-ecr-staging.outputs.registry }}
run: |
functionName=$([ "$LAMBDA_NAME" == "submission" ] && echo "Submission" || echo "$LAMBDA_NAME")
aws lambda update-function-code --function-name $functionName --image-uri $ECR_REGISTRY/$LAMBDA_NAME-lambda:$IMAGE_TAG
shell: bash

- name: Logout of Staging Amazon ECR
if: always()
run: docker logout ${{ steps.login-ecr-staging.outputs.registry }}
shell: bash
45 changes: 45 additions & 0 deletions .github/workflows/tag-and-push-lambda-images/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Tag and push Lambda images

inputs:
aws-role-to-assume:
required: true
aws-role-session-name:
required: true
aws-region:
required: true
lambda-name:
required: true
image-tag:
required: true

runs:
using: "composite"
steps:
- name: Configure AWS credentials using OIDC
uses: aws-actions/configure-aws-credentials@010d0da01d0b5a38af31e9c3470dbfdabdecca3a # v4.0.1
with:
role-to-assume: ${{ inputs.aws-role-to-assume }}
role-session-name: ${{ inputs.aws-role-session-name }}
aws-region: ${{ inputs.aws-region }}

- name: Login to Staging Amazon ECR
id: login-ecr-staging
uses: aws-actions/amazon-ecr-login@v2

- name: Tag and push docker images
env:
LAMBDA_NAME: ${{ inputs.lambda-name }}
IMAGE_TAG: ${{ inputs.image-tag }}
ECR_REGISTRY: ${{ steps.login-ecr-staging.outputs.registry }}
run: |
REPOSITORY_NAME=$LAMBDA_NAME-lambda
docker tag $REPOSITORY_NAME $ECR_REGISTRY/$REPOSITORY_NAME:$IMAGE_TAG
docker tag $REPOSITORY_NAME $ECR_REGISTRY/$REPOSITORY_NAME:latest
docker push $ECR_REGISTRY/$REPOSITORY_NAME:$IMAGE_TAG
docker push $ECR_REGISTRY/$REPOSITORY_NAME:latest
craigzour marked this conversation as resolved.
Show resolved Hide resolved
shell: bash

- name: Logout of Staging Amazon ECR
if: always()
run: docker logout ${{ steps.login-ecr-staging.outputs.registry }}
shell: bash
Loading
Loading