Initial release (#11)

* Initial upload * README parameters and repo cleanup * Adds Actions CI workflows (#1) * Adds Actions CI workflows * Corrects Go workflows * Cleans up unrequired comments * Adds some unit tests (#2) * Adds some unit tests * Excludes Gosec variable path check As not relevant for execution environment * Adds initial PR deploy workflow (#3) * Adds initial PR deploy workflow * Corrects if event statement * Updates `sed` syntax to GNU version * You must construct additional `\`s * Switches to non-container build * Updates tags and adds test input image * Corrects CF output query strings * Corrects output query quotes again * Adds naming for CloudFront redirect Function * Applies consistent `PR` capitalisation * Adds PR cleanup workflow (#4) * Adds PR cleanup workflow * Adds `CloudFrontCachePolicyName` * Corrects cleanup workflow (#5) * Adds required SAM stack args (#6) * Removes unsupported `--role-arn` from SAM destroy (#7) * Removes `sam delete` confirmation requirement (#8) * Adds invalid arg exit to Python script * README diagram improvements * Adds new stack parameters to README * Adds additions README info * Adds OpenGraph note * Passes CloudFront domain to function when `NoDomain` is set (#10) * Passes CloudFront domain to function When NoDomain is set * Adds repo name tag and JPEG upload to PR deploy * Adds license * Adds ACM region note to parameters * Updates README to new example Redeployed after closing #9 * Removed redundant tag * Updates tests with `http-equiv` tag removal * Adds Slack notification for PR deployment * Corrects Slack payload * Corrects Slack payload again * Adds repo info to Slack notification * Adds emoji to Slack message * Whoops * Adds cleanup workflow notification
Phuurl · Apr 29, 2022 · eec1098 · eec1098
1 parent 7750831
commit eec1098
Show file tree

Hide file tree

Showing 20 changed files with 1,478 additions and 1 deletion.
diff --git a/.ci/empty-versioned-bucket.py b/.ci/empty-versioned-bucket.py
@@ -0,0 +1,13 @@
+#!/usr/bin/env python3
+
+import sys
+
+import boto3
+
+if len(sys.argv) != 2:
+    print("Missing / invalid arguments\nRequires bucket name as 1st and only script argument.")
+    sys.exit(1)
+bucket_name = sys.argv[1]
+s3 = boto3.resource('s3')
+bucket = s3.Bucket(bucket_name)
+bucket.object_versions.delete()
diff --git a/.ci/samconfig.toml.template b/.ci/samconfig.toml.template
@@ -0,0 +1,13 @@
+version = 0.1
+[ci]
+[ci.deploy]
+[ci.deploy.parameters]
+stack_name = "{}"
+s3_bucket = "{}"
+s3_prefix = "{}"
+region = "eu-west-1"
+confirm_changeset = false
+capabilities = "CAPABILITY_IAM"
+parameter_overrides = "{}"
+image_repositories = []
+tags = "{}"
diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml
@@ -0,0 +1,105 @@
+name: Checks and PR stack deploy
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+
+jobs:
+  cloudformation-check:
+    runs-on: ubuntu-latest
+    container:
+      image: us-central1-docker.pkg.dev/phurl-public-docker/github-actions/actions-cfn-lint:latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v2
+      - name: Lint CloudFormation templates
+        run: |
+          cfn-lint --version
+          cfn-lint --include-checks I --info -t template.yaml
+  go-check:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Go
+        uses: actions/setup-go@v2
+        with:
+          go-version: 1.17
+      - name: Get tools
+        run: |
+          go install honnef.co/go/tools/cmd/staticcheck@latest
+          curl -sfL https://raw.githubusercontent.com/securego/gosec/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.11.0
+      - name: Run Staticcheck
+        run: |
+          staticcheck -f stylish ./...
+        working-directory: ingest-fn
+      - name: Run Gosec
+        run: |
+          gosec -exclude=G304 -verbose text ./...
+        working-directory: ingest-fn
+      - name: Run tests
+        run: |
+          go test -v
+        working-directory: ingest-fn
+  deploy-pr-stack:
+    if: github.event_name == 'pull_request'
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write
+      contents: read
+    timeout-minutes: 15
+    needs:
+      - cloudformation-check
+      - go-check
+    steps:
+      - run: sleep 5
+      - uses: actions/checkout@v2
+      - uses: aws-actions/[email protected]
+        with:
+          role-to-assume: ${{ secrets.ACTIONS_ROLE_ARN }}
+          role-duration-seconds: 900
+          aws-region: eu-west-1
+      - name: Place SAM config
+        run: |
+          cp .ci/samconfig.toml.template samconfig.toml
+          sed -i -e "s|stack_name = \"{}\"|stack_name = \"GITHUB-Phuurl-img-site-PR-${PRNUM}\"|g" samconfig.toml
+          sed -i -e "s|s3_bucket = \"{}\"|s3_bucket = \"${SAM_BUCKET}\"|g" samconfig.toml
+          sed -i -e "s|s3_prefix = \"{}\"|s3_prefix = \"GITHUB-Phuurl-img-site-PR-${PRNUM}\"|g" samconfig.toml
+          sed -i -e "s|parameter_overrides = \"{}\"|parameter_overrides = \"UploadBucketName=\\\\\"github-phuurl-img-site-pr-upload-${PRNUM}\\\\\" EmailNotificationEnabled=\\\\\"false\\\\\" NotificationEmail=\\\\\"x\\\\\" SiteName=\\\\\"GITHUB-Phuurl-img-site-PR-${PRNUM}\\\\\" NoDomain=\\\\\"true\\\\\" Domain=\\\\\"x\\\\\" CertArn=\\\\\"x\\\\\" CloudFrontRedirectFunctionName=\\\\\"GITHUB_Phuurl_img_site_PR_${PRNUM}_index_redirect_func\\\\\" CloudFrontCachePolicyName=\\\\\"GITHUB-Phuurl-img-site-PR-${PRNUM}-cache-policy\\\\\" CreateUploadIamUser=\\\\\"false\\\\\"\"|g" samconfig.toml
+          sed -i -e "s|tags = \"{}\"|tags = \"project=\\\\\"github\\\\\" repo=\\\\\"img-site\\\\\" pr=\\\\\"${PRNUM}\\\\\"\"|g" samconfig.toml
+          cat samconfig.toml
+        env:
+          PRNUM: ${{ github.event.number }}
+          SAM_BUCKET: ${{ secrets.SAM_BUCKET }}
+      - name: SAM build
+        run: |
+          sed -i -e "s|linux-arm64|linux-amd64|g" ingest-fn/Makefile
+          sam build
+      - name: Slack notify
+        run: |
+          curl -X POST -H 'Content-type: application/json' --data "{\"text\": \":shipit: PR deployment for \`Phuurl/img-site/#${PRNUM}\`\"}" ${SLACK_WEBHOOK}
+        env:
+          PRNUM: ${{ github.event.number }}
+          SLACK_WEBHOOK: ${{ secrets.SLACK_PR_DEPLOYMENT_WEBHOOK }}
+        continue-on-error: true
+      - name: SAM deploy
+        run: |
+          sam deploy --role-arn ${DEPLOY_ROLE} --config-env ci
+        env:
+          DEPLOY_ROLE: ${{ secrets.DEPLOY_ROLE_ARN }}
+      - name: Upload static assets
+        run: |
+          HostingBucketName=$(aws cloudformation describe-stacks --stack-name "GITHUB-Phuurl-img-site-PR-${PRNUM}" --query "Stacks[0].Outputs[?OutputKey=='HostingBucketName'].OutputValue" --output text --region eu-west-1)
+          aws s3 cp error.html s3://${HostingBucketName}/
+          aws s3 cp robots.txt s3://${HostingBucketName}/
+        env:
+          PRNUM: ${{ github.event.number }}
+      - name: Upload input images
+        run: |
+          UploadBucketName=$(aws cloudformation describe-stacks --stack-name "GITHUB-Phuurl-img-site-PR-${PRNUM}" --query "Stacks[0].Outputs[?OutputKey=='UploadBucketName'].OutputValue" --output text --region eu-west-1)
+          aws s3 cp ingest-fn/.test-resources/500x500.png s3://${UploadBucketName}/
+          aws s3 cp ingest-fn/.test-resources/500x500.jpg s3://${UploadBucketName}/
+        env:
+          PRNUM: ${{ github.event.number }}
diff --git a/.github/workflows/pr-cleanup.yml b/.github/workflows/pr-cleanup.yml
@@ -0,0 +1,58 @@
+name: PR Cleanup
+
+on:
+  pull_request:
+    types:
+      - closed
+
+jobs:
+  cleanup-pr-stack:
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write
+      contents: read
+    timeout-minutes: 15
+    steps:
+      - run: sleep 5
+      - uses: actions/checkout@v2
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v3
+        with:
+          python-version: "3.10"
+      - uses: aws-actions/[email protected]
+        with:
+          role-to-assume: ${{ secrets.ACTIONS_ROLE_ARN }}
+          role-duration-seconds: 900
+          aws-region: eu-west-1
+      - name: Install boto3 for Python script
+        run: |
+          pip3 install boto3
+      - name: Place SAM config
+        run: |
+          cp .ci/samconfig.toml.template samconfig.toml
+          sed -i -e "s|stack_name = \"{}\"|stack_name = \"GITHUB-Phuurl-img-site-PR-${PRNUM}\"|g" samconfig.toml
+          sed -i -e "s|s3_bucket = \"{}\"|s3_bucket = \"${SAM_BUCKET}\"|g" samconfig.toml
+          sed -i -e "s|s3_prefix = \"{}\"|s3_prefix = \"GITHUB-Phuurl-img-site-PR-${PRNUM}\"|g" samconfig.toml
+          sed -i -e "s|parameter_overrides = \"{}\"|parameter_overrides = \"UploadBucketName=\\\\\"github-phuurl-img-site-pr-upload-${PRNUM}\\\\\" EmailNotificationEnabled=\\\\\"false\\\\\" NotificationEmail=\\\\\"x\\\\\" SiteName=\\\\\"GITHUB-Phuurl-img-site-PR-${PRNUM}\\\\\" NoDomain=\\\\\"true\\\\\" Domain=\\\\\"x\\\\\" CertArn=\\\\\"x\\\\\" CloudFrontRedirectFunctionName=\\\\\"GITHUB_Phuurl_img_site_PR_${PRNUM}_index_redirect_func\\\\\" CreateUploadIamUser=\\\\\"false\\\\\"\"|g" samconfig.toml
+          sed -i -e "s|tags = \"{}\"|tags = \"project=\\\\\"github\\\\\" pr=\\\\\"${PRNUM}\\\\\"\"|g" samconfig.toml
+          cat samconfig.toml
+        env:
+          PRNUM: ${{ github.event.number }}
+          SAM_BUCKET: ${{ secrets.SAM_BUCKET }}
+      - name: Empty buckets
+        run: |
+          HostingBucketName=$(aws cloudformation describe-stacks --stack-name "GITHUB-Phuurl-img-site-PR-${PRNUM}" --query "Stacks[0].Outputs[?OutputKey=='HostingBucketName'].OutputValue" --output text --region eu-west-1)
+          UploadBucketName=$(aws cloudformation describe-stacks --stack-name "GITHUB-Phuurl-img-site-PR-${PRNUM}" --query "Stacks[0].Outputs[?OutputKey=='UploadBucketName'].OutputValue" --output text --region eu-west-1)
+          python3 .ci/empty-versioned-bucket.py ${HostingBucketName}
+          python3 .ci/empty-versioned-bucket.py ${UploadBucketName}
+        env:
+          PRNUM: ${{ github.event.number }}
+      - name: SAM delete
+        run: |
+          sam delete --config-env ci --no-prompts
+      - name: Slack notify
+        run: |
+          curl -X POST -H 'Content-type: application/json' --data "{\"text\": \":dusty_stick: Cleanup for PR \`Phuurl/img-site/#${PRNUM}\`\"}" ${SLACK_WEBHOOK}
+        env:
+          PRNUM: ${{ github.event.number }}
+          SLACK_WEBHOOK: ${{ secrets.SLACK_PR_DEPLOYMENT_WEBHOOK }}
diff --git a/.gitignore b/.gitignore
@@ -0,0 +1,203 @@
+
+# Created by https://www.toptal.com/developers/gitignore/api/go,linux,macos,sam+config,visualstudiocode,jetbrains+all
+# Edit at https://www.toptal.com/developers/gitignore?templates=go,linux,macos,sam+config,visualstudiocode,jetbrains+all
+
+### Go ###
+# If you prefer the allow list template instead of the deny list, see community template:
+# https://github.com/github/gitignore/blob/main/community/Golang/Go.AllowList.gitignore
+#
+# Binaries for programs and plugins
+*.exe
+*.exe~
+*.dll
+*.so
+*.dylib
+
+# Test binary, built with `go test -c`
+*.test
+
+# Output of the go coverage tool, specifically when used with LiteIDE
+*.out
+
+# Dependency directories (remove the comment below to include it)
+# vendor/
+
+# Go workspace file
+go.work
+
+### Go Patch ###
+/vendor/
+/Godeps/
+
+### JetBrains+all ###
+# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio, WebStorm and Rider
+# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
+
+# User-specific stuff
+.idea/**/workspace.xml
+.idea/**/tasks.xml
+.idea/**/usage.statistics.xml
+.idea/**/dictionaries
+.idea/**/shelf
+
+# AWS User-specific
+.idea/**/aws.xml
+
+# Generated files
+.idea/**/contentModel.xml
+
+# Sensitive or high-churn files
+.idea/**/dataSources/
+.idea/**/dataSources.ids
+.idea/**/dataSources.local.xml
+.idea/**/sqlDataSources.xml
+.idea/**/dynamic.xml
+.idea/**/uiDesigner.xml
+.idea/**/dbnavigator.xml
+
+# Gradle
+.idea/**/gradle.xml
+.idea/**/libraries
+
+# Gradle and Maven with auto-import
+# When using Gradle or Maven with auto-import, you should exclude module files,
+# since they will be recreated, and may cause churn.  Uncomment if using
+# auto-import.
+# .idea/artifacts
+# .idea/compiler.xml
+# .idea/jarRepositories.xml
+# .idea/modules.xml
+# .idea/*.iml
+# .idea/modules
+# *.iml
+# *.ipr
+
+# CMake
+cmake-build-*/
+
+# Mongo Explorer plugin
+.idea/**/mongoSettings.xml
+
+# File-based project format
+*.iws
+
+# IntelliJ
+out/
+
+# mpeltonen/sbt-idea plugin
+.idea_modules/
+
+# JIRA plugin
+atlassian-ide-plugin.xml
+
+# Cursive Clojure plugin
+.idea/replstate.xml
+
+# SonarLint plugin
+.idea/sonarlint/
+
+# Crashlytics plugin (for Android Studio and IntelliJ)
+com_crashlytics_export_strings.xml
+crashlytics.properties
+crashlytics-build.properties
+fabric.properties
+
+# Editor-based Rest Client
+.idea/httpRequests
+
+# Android studio 3.1+ serialized cache file
+.idea/caches/build_file_checksums.ser
+
+### JetBrains+all Patch ###
+# Ignores the whole .idea folder and all .iml files
+# See https://github.com/joeblau/gitignore.io/issues/186 and https://github.com/joeblau/gitignore.io/issues/360
+
+.idea/*
+
+# Reason: https://github.com/joeblau/gitignore.io/issues/186#issuecomment-249601023
+
+*.iml
+modules.xml
+.idea/misc.xml
+*.ipr
+
+# Sonarlint plugin
+.idea/sonarlint
+
+### Linux ###
+*~
+
+# temporary files which can be created if a process still has a handle open of a deleted file
+.fuse_hidden*
+
+# KDE directory preferences
+.directory
+
+# Linux trash folder which might appear on any partition or disk
+.Trash-*
+
+# .nfs files are created when an open file is removed but is still being accessed
+.nfs*
+
+### macOS ###
+# General
+.DS_Store
+.AppleDouble
+.LSOverride
+
+# Icon must end with two \r
+Icon
+
+
+# Thumbnails
+._*
+
+# Files that might appear in the root of a volume
+.DocumentRevisions-V100
+.fseventsd
+.Spotlight-V100
+.TemporaryItems
+.Trashes
+.VolumeIcon.icns
+.com.apple.timemachine.donotpresent
+
+# Directories potentially created on remote AFP share
+.AppleDB
+.AppleDesktop
+Network Trash Folder
+Temporary Items
+.apdisk
+
+### SAM+config ###
+# Ignore build directories for the AWS Serverless Application Model (SAM)
+# Info: https://aws.amazon.com/serverless/sam/
+# Docs: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-reference.html
+
+**/.aws-sam
+
+### SAM+config Patch ###
+# SAM config - exclude this file if sharing publicly
+samconfig.toml
+
+### VisualStudioCode ###
+.vscode/*
+!.vscode/settings.json
+!.vscode/tasks.json
+!.vscode/launch.json
+!.vscode/extensions.json
+!.vscode/*.code-snippets
+
+# Local History for Visual Studio Code
+.history/
+
+# Built Visual Studio Code Extensions
+*.vsix
+
+### VisualStudioCode Patch ###
+# Ignore all local history of files
+.history
+.ionide
+
+# Support for Project snippet scope
+
+# End of https://www.toptal.com/developers/gitignore/api/go,linux,macos,sam+config,visualstudiocode,jetbrains+all