Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quickstart #1

Open
wants to merge 13 commits into
base: fork
Choose a base branch
from
Open

Quickstart #1

wants to merge 13 commits into from

Conversation

bjrbhre
Copy link
Owner

@bjrbhre bjrbhre commented Mar 20, 2019

Purpose

Deploy a JWT-secured JSON-LD Rest API + React admin backoffice in <1h 20 minutes 🚀
See it live:

Just Do It 🔥

#! /usr/bin/env bash

set -e

[ -z "$AWS_ACCESS_KEY_ID" ] && echo "AWS_ACCESS_KEY_ID not defined" && exit 1
[ -z "$AWS_SECRET_ACCESS_KEY" ] && echo "AWS_SECRET_ACCESS_KEY not defined" && exit 1
[ -z "$HEROKU_TEAM" ] && echo "HEROKU_TEAM not defined" && exit 1

git clone -b 'quickstart' -o fork [email protected]:bjrbhre/api-platform.git quickstart
cd quickstart
make start && make fixtures

JWT_DIR=provision/jwt
JWT_PASSPHRASE=$(openssl rand -base64 32) 

make provision
make deploy

Context

This quick start is based on API Platform, adding the following features:

  • API entities: additional requirements for easier creation of API Entities
  • Deployment: config to deploy on Heroku (API) and AWS/S3 (admin)
  • Security: requirements & config to add JWT in local & deployed environment
  • Automation: Makefile to automate install, provision, deploy

This is mainly based on Getting Started with API Platform and related documentations. API Platform has a 🕷as a logo and is built on top of Symfony. In their own word

REST and GraphQL framework to build modern API-driven projects

API

Among other things, we have the following features out-of-the-box:

  • API + admin / client apps boilerplates
  • Docker & Helm setup
  • Swagger doc + "Try it" page

api-demo

Admin Backoffice

The admin app is particularly interesting. It leverages the JSON-LD exposed by the API to auto-discover resources and expose them using React-admin and Create React App.

admin-demo

How To

Below you will find links to commits from this fork. In each commit & commit message you will find more details about what commands to type, doc to read, changes to implement.

Fast Foward ⏩

If you just want to go straight to the point, clone the fork and run make.

Clone the fork

git clone -b 'quickstart' -o fork [email protected]:bjrbhre/api-platform.git quickstart
cd quickstart
git remote add -f upstream [email protected]:api-platform/api-platform.git

Start on localhost

make start && make fixtures

When the containers are up, you should now have access to:

And you should be able to connect with fixture'd credentials:

To get a JWT token

Provision & Deploy

Provision Heroku app and AWS/S3 bucket & Deploy

JWT_DIR=provision/jwt 
JWT_PASSPHRASE=$(openssl rand -base64 32) 
make provision
make deploy

Test Live

Create a user

make api/user

If you want to grant ROLE_ADMIN role to your user:

heroku run ./api/bin/console fos:user:promote $EMAIL ROLE_ADMIN

You should now be able to connect to the API hosted on Heroku or directly to the admin backoffice hosted on AWS/S3:

  • API on https://${HEROKU_TEAM}-api-staging.herokuapp.com
  • Admin on http://${HEROKU_TEAM}-admin-staging.s3-website.eu-west-3.amazonaws.com

Detailed Steps 🔍

If you want to understand the different points addressed on top of API Platform distribution, you can reproduce this step-by-step guide.

Prerequisites ✅

The following commands should succeed

which git
which docker
which docker-compose
which aws
which heroku

For deployment:

  • make sure you have an AWS user w/ access keys and AmazonS3FullAccess, see doc
  • make sure you create a Heroku team, see doc
  • export those variables, consider using direnv
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export HEROKU_TEAM=...

Checkout API Platform distribution 🎉

We suppose we will work in a quickstart folder.
Checkout the latest stable version of API Platform.

git clone -b 'v2.3.6' -o upstream [email protected]:api-platform/api-platform.git quickstart
cd quickstart
git checkout -b quickstart

You can also download an archive of any release here.

If you plan to use this as a boilerplate for your own project and don't care about upstream updates, you can squash all commits or reset git setup:

rm -rf .git
git init
git add .
git commit -m ":tada: Init from [API Platform v2.3.6](https://github.com/api-platform/api-platform/releases/tag/v2.3.6)"

You can also consider adding it as a sub-folder of your already-existing project with git-subtree.

Build 🐳

Build it... and go take a coffee ☕️

docker-compose up -d

When the containers are up, you should now have access to:

API entities 🗃

Deployment 🚀

Security 🔒

Deployment automation ♻️ 🚀

Next Steps 🤔

Add your own models 🗃

Leverage the Symfony MakerBundle to quickly create your own models:

alias sf='docker-compose exec php ./bin/console'
sf make:entity --api-resource
sf doctrine:migration:diff

See:

Generate a client app 🎨

API Platform also has on-the-shelf tooling to setup an Client app boilerplate

docker-compose exec client generate-api-platform-client

🐛 It fails due to

Error: Cannot find module 'lodash'

See https://github.com/api-platform/client-generator/issues/108

I cannot remember how I made it work on api-platform/api-platform@6f55803 🙈. If you do, then, as explained by the console output, import and inject the generated routes and reducers in the App. See api-platform/api-platform@7978c52

To Do 👷

Models

Features

Deployment

Misc

- maker-bundle to easily create Entities
- validator and annotations to enforce requirements on Entities
- migrations
- behat for E2E testing (needs some adjustements to make it work)

```
docker-compose exec php composer require symfony/maker-bundle --dev
docker-compose exec php composer require symfony/validator doctrine/annotations
docker-compose exec php composer require migrations
docker-compose exec php composer require --dev behat/behat
docker-compose exec php vendor/bin/behat --init
```
bjrbhre added 12 commits March 22, 2019 17:06
```
docker-compose exec php composer require symfony/translation
docker-compose exec  php composer require symfony/apache-pack
```
Makefile target to create app
```
provision/api/app:
	heroku apps:create \
		--team $(HEROKU_TEAM) \
		--region $(HEROKU_REGION) \
		--remote $(DEPLOYMENT_ENV) \
		--buildpack https://github.com/negativetwelve/heroku-buildpack-subdir \
		--addons heroku-postgresql:hobby-dev \
		$(API_APP_NAME)
```

Makefile target to setup environment
```
provision/api/environment: provision/api/app
	heroku config:set -a $(API_APP_NAME) \
		APP_ENV=prod \
		APP_SECRET=$(shell openssl rand -base64 32) \
		CORS_ALLOW_ORIGIN=$(ADMIN_URL) \
		TRUSTED_HOSTS=$(API_HOST)
```
Mostly inspired from [doc](https://api-platform.com/docs/core/jwt/).
- use Lexik for JWT authentication
- use FOSUserBundle for User entity
- FOSUserBundle config depends on SwiftMailer
- add doctrine fixtures to provision users

```
docker-compose exec php composer require "lexik/jwt-authentication-bundle"
docker-compose exec php composer require friendsofsymfony/user-bundle
docker-compose exec php composer require swiftmailer-bundle
docker-compose exec php composer require --dev doctrine/doctrine-fixtures-bundle
```
- table is `fos_user` to avoid conflict w/ Postgres `user`
- enable users by default
- add ACLs on User
- fixtures for admin & user
Issue is that JWT keys cannot be store on the filesystem.
Proposal is to read them from ENV.
Start on localhost
```
make start fixtures
[email protected] make token
[email protected] make token
```

Provision & Deploy
```
JWT_DIR=provision/jwt JWT_PASSPHRASE=$(openssl rand -base64 32) make provision  # to provision deployment env
make deploy
```

Cleaning
```
make clean
make clean/all
```

Destroy (requires confirmation)
```
make destroy/api  # interactive confirmation
FORCE="--force" make destroy/admin  # requires --force to confirm
```
@bjrbhre
Copy link
Owner Author

bjrbhre commented Mar 25, 2019

Using subtree:

ifndef SUBTREE
SUBTREE=api-platform
endif

deploy/api:
	cd .. && git push $(DEPLOYMENT_ENV) `git subtree split --prefix ${SUBTREE} HEAD`:master $(FORCE)

https://brettdewoody.com/deploying-a-heroku-app-from-a-subdirectory/

@bjrbhre
Copy link
Owner Author

bjrbhre commented Mar 26, 2019

Secured Infra 🔒

How to Provision 🔥

AWS_KMS_KEY_ID=... \
DB_INSTANCE_CLASS=... \
ALLOCATED_STORAGE=... \
MASTER_USER_PASSWORD=$(openssl rand -base64 32) \
make provision/aws
AWS_CLOUDFRONT_DISTRIBUTION_ID=...\ 
API_HOST=api-staging.${CUSTOME_DOMAIN} \
make deploy/admin

Detailed Procedure 🔍

The following security measure can be added to your deployment:

  • provision an encrypted DB on AWS/RDS
  • enforce SSL connection to this DB
  • deploy API / Admin on custom domain w/ SSL support
  • redirect HTTP to HTTPS

Below you can find a draft of Makefile targets that can do 80% of the job + manual instructions for the remaining 20%.

Create Security Group

Used to define the ingress rules for the DB access

provision/$(DEPLOYMENT_ENV)/aws/ec2/security_group:
	aws --region $(AWS_REGION) ec2 create-security-group \
		--group $(API_APP_NAME) \
		--description "Security group for $(API_APP_NAME)"
	$(eval SECURITY_GROUP_ID=`aws ec2 describe-security-groups --group-names $(API_APP_NAME) | jq -r .SecurityGroups[0].GroupId`)
	aws ec2 authorize-security-group-ingress \
		--group-id "$(SECURITY_GROUP_ID)" \
		--ip-permissions IpProtocol=tcp,FromPort=5432,ToPort=5432,IpRanges='[{Description=PSQL/Public,CidrIp=0.0.0.0/0}]',UserIdGroupPairs="[{Description=PSQL/SG,GroupId=$(SECURITY_GROUP_ID)}]"
	mkdir -p $@

provision/$(DEPLOYMENT_ENV)/aws/ec2: provision/$(DEPLOYMENT_ENV)/aws/ec2/security_group
	touch $@

Create RDS/Parameter Group

Used to add custom params to the DB config (require SSL)

provision/aws/rds/db-parameter-group:
	aws --region $(AWS_REGION) rds create-db-parameter-group \
		--db-parameter-group-name secured-postgres10 \
		--db-parameter-group-family postgres10 \
		--description "Add secured params"
	aws --region $(AWS_REGION) rds modify-db-parameter-group \
		--db-parameter-group-name secured-postgres10 \
		--parameters ParameterName=rds.force_ssl,ParameterValue=1,ApplyMethod=immediate
	mkdir -p $@

Create Database Instance

⚠️ Request Certificate in your AWS_REGION (ex. eu-west-3 for Paris) on AWS/CloudFront and store its ARN in AWS_KMS_KEY_ID.

provision/$(DEPLOYMENT_ENV)/aws/rds: provision/$(DEPLOYMENT_ENV)/aws/rds/create-db-instance provision/$(DEPLOYMENT_ENV)/aws/ec2/security_group
	$(eval SECURITY_GROUP_ID=`aws ec2 describe-security-groups --group-names $(API_APP_NAME) | jq -r .SecurityGroups[0].GroupId`)
	aws --region $(AWS_REGION) rds create-db-instance \
		--engine postgres \
		--engine-version 10.6 \
		--db-instance-class $(DB_INSTANCE_CLASS) \
		--allocated-storage $(ALLOCATED_STORAGE) \
		--db-instance-identifier $(API_APP_NAME)-`uuidgen` \
		--db-name api \
		--port 5432 \
		--master-username $(HEROKU_TEAM)_api_$(DEPLOYMENT_ENV) \
		--master-user-password $(MASTER_USER_PASSWORD) \
		--publicly-accessible \
		--vpc-security-group-ids "$(SECURITY_GROUP_ID)" \
		--db-parameter-group-name secured-postgres10 \
		--kms-key-id ${AWS_KMS_KEY_ID} \
		--storage-encrypted

Recommandation for prod instance specifications:

DB_INSTANCE_CLASS=db.t2.medium 
ALLOCATED_STORAGE=100

When created & available, you can test the connection:

# list instances
aws rds describe-db-instances | \
    jq -r '.DBInstances[]| [.Endpoint.Address, .StorageEncrypted] | join(", ")'

# connect w/o SSL (should fail)
psql -h $DB_HOST -p 5432 "user=$DB_USER dbname=$DB_NAME sslmode=disable"

# forcing SSL
psql -h $DB_HOST -p 5432 "user=$DB_USER dbname=$DB_NAME sslmode=require"

Related doc:

Provision SSL Certificate

Use Let's Encrypt and certbot

brew install certbot
sudo certbot certonly --manual
heroku certs:add /etc/letsencrypt/live/domain/fullchain.pem /etc/letsencrypt/live/domain/privkey.pem

Then add you custom domain:

heroku domains:add api-staging.${CUSTOM_DOMAIN}

And add the related CNAME record in AWS/Route53

See this post

Update Heroku config

heroku config:set CORS_ALLOW_ORIGIN="^https://(admin|client)-staging.${CUSTOM_DOMAIN}$"
heroku config:set DATABASE_URL=postgres://user:password@host:5432/api?sslmode=require
heroku config:set TRUSTED_HOSTS=api-staging.${CUSTOM_DOMAIN}

Cloudfront distribution

⚠️ Request Certificate in us-east-1. Go to AWS/CloudFront

Create a Web distribution

  • Origin Domain Name: the endpoint url of the bucket ${HEROKU_TEAM}-admin-${DEPLOYMENT_ENV}.s3-website.eu-west-3.amazonaws.com
  • Default Cache Behavior Settings:
    • select Redirect HTTP to HTTPS
  • Distribution Settings
    • Price Class: select Use Only U.S., Canada and Europe
    • Alternate Domain Names: admin-${DEPLOYMENT_ENV}.${CUSTOM_DOMAIN}
    • SSL Certificate: Custom SSL Certificate and chose the wildcard
    • Default Root Object: index.html

Once created, configure custom error response
Screenshot 2019-03-25 19 32 50

DNS Record for Cloudfront

Wait for the Cloudfront distribution to be provisionned.
Go to AWS/Route53.
Create a new A record entry with using Alias and Alias Target equal to the Cloudfront distribution.

Known issues

  • 🐛 Postgres sslmode=verify-full on Heroku StackOverflow / doc
  • ⚠️ DB version in postgres:10-alpine Container is 10.7 vs. 10.6-R1 on AWS/RDS

Makefile adjustments

# override API_HOST to deploy Admin on custom domain
ifndef API_HOST
API_HOST=$(HEROKU_TEAM)-api-$(DEPLOYMENT_ENV).herokuapp.com
endif

# Fix hardcoded value of S3 bucket in `provision/admin`
provision/admin:
	aws s3 mb --region $(AWS_REGION) $(ADMIN_BUCKET_URL)
	aws s3 website $(ADMIN_BUCKET_URL) --index-document index.html --error-document index.html
	mkdir -p $@

# Fix push to refs/heads/master in deploy/api
deploy/api:
	cd .. && git push $(DEPLOYMENT_ENV) `git subtree split --prefix ${SUBTREE} HEAD`:refs/heads/master $(FORCE)

@bjrbhre
Copy link
Owner Author

bjrbhre commented Mar 29, 2019

Security 🔒

Require ROLE_USER to access API doc

# security.yml
security:
    access_control:
        - { path: ^/login, roles: IS_AUTHENTICATED_ANONYMOUSLY, requires_channel: https }
        - { path: ^/contexts/Error, roles: IS_AUTHENTICATED_ANONYMOUSLY, requires_channel: https }
        - { path: ^/, roles: ROLE_USER, requires_channel: https }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant