Golang Serverless with local development

Description

Repository presenting example setup for Serverless Golang project for AWS.

IaC

Infrastructure is generated by the Serverless framework using Cloudformation service under the hood. Infrastructure code is in YAML files with serverless.yml being the entry.

Only serverless services with pricing determined by usage are created:

• API Gateway
• Lambda
• S3
• DynamoDB

What it does?

The repository contains two Lambda functions.

• The first function is triggered by object creation in S3 with key prefix uploads/ and suffix .csv. When triggered it performs simple ETL (Extract, Transform, Load) from CSV to DynamoDB in batches.
• The second function is triggered by HTTP requests and simply returns an item from DynamoDB by specified id.

Local development

To allow full local development combination of couple technics is utilized.

• AWS services are emulated by Localstack in a container defined in Docker-Compose (S3, DynamoDB).
• AWS endpoint resolver is redirected to the Localstack Docker container when run offline (check: AwsEndpointResolverFactory)
• the serverless-offline plugin is used to invoke HTTP lambdas on HTTP events on localhost
• Functions triggered by other events (s3:ObjectCreated) are invoked from CLI by using sls invoke local command and passing JSON event.

Setup

Prerequisites

• Go installed
• Node.JS installed
• Docker installed

Initial setup

# install serverless packages
$ npm i

# install golang packages
$ go mod download

# copy example `.env` file
$ cp .env.example .env

Running locally

# start localstack
$ make docker-start

# import file with seed data to local s3
$ make s3-upload-csv

# run import lambda
$ make sls-invoke-s3

# start http lambdas (serverless-offline)
$ make offline

Testing

Unit tests are using Testify assertion library with mocks generated from interfaces using Mockery

# unit tests
$ go test ./...

# unit test with verbose output
$ go test ./... -v

Commands

Mocks

# install mockery
$ make mockery-install

# generate mocks
$ make mockery

Deployment

sls deploy --aws-profile priv --stage dev

Local DynamoDB helper commands

# getting item from dynamodb by id
$ make dynamo-get id=12

# scan
$ make dynamo-scan

# count
$ make dynamo-count

# delete table
$ make dynamo-delete-table:

# create table
$ make dynamo-create-table:

# clear table (delete and recreate)
$ make dynamo-clear