From 88705b4d2cd886f9bd6658541b8e2d84944d7660 Mon Sep 17 00:00:00 2001 From: Ryan Ling Date: Tue, 26 Jan 2021 23:11:49 +1100 Subject: [PATCH] Uplift template READMEs (#334) Co-authored-by: Toby Heighway --- .changeset/young-terms-sleep.md | 5 ++ README.md | 2 +- template/express-rest-api/README.md | 59 +++++++++++++++- template/greeter/README.md | 94 +++++++++++++++++++++++--- template/koa-rest-api/README.md | 68 +++++++++++++++++-- template/lambda-sqs-worker/README.md | 86 ++++++++++++++++++++--- template/oss-npm-package/README.md | 25 +++++-- template/private-npm-package/README.md | 29 ++++++-- 8 files changed, 330 insertions(+), 38 deletions(-) create mode 100644 .changeset/young-terms-sleep.md diff --git a/.changeset/young-terms-sleep.md b/.changeset/young-terms-sleep.md new file mode 100644 index 000000000..cfde425f5 --- /dev/null +++ b/.changeset/young-terms-sleep.md @@ -0,0 +1,5 @@ +--- +'skuba': patch +--- + +**template:** Uplift READMEs diff --git a/README.md b/README.md index ab8ca966d..d2ee468fd 100644 --- a/README.md +++ b/README.md @@ -187,7 +187,7 @@ This initialises a new directory and Git repository. [aws lambda]: https://tech-strategy.ssod.skinfra.xyz/docs/v1/technology.html#lambda [koa]: https://koajs.com/ [resource api]: https://tech-strategy.ssod.skinfra.xyz/docs/v1/components.html#resource-api -[seek-jobs]: https://github.com/SEEK-Jobs +[seek-jobs]: https://github.com/orgs/seek-jobs/sso [seek-oss]: https://github.com/seek-oss [semantic-release]: https://github.com/semantic-release/semantic-release/ [serverless]: https://serverless.com/ diff --git a/template/express-rest-api/README.md b/template/express-rest-api/README.md index 9fd2e548b..4f4cf3ee1 100644 --- a/template/express-rest-api/README.md +++ b/template/express-rest-api/README.md @@ -2,12 +2,31 @@ [![Powered by skuba](https://img.shields.io/badge/🤿%20skuba-powered-009DC4)](https://github.com/seek-oss/skuba) +Next steps: + +1. [ ] Finish templating if this was skipped earlier: + + ```shell + yarn skuba configure + ``` + +2. [ ] Create a new repository in the appropriate GitHub organisation. +3. [ ] Add the repository to BuildAgency; + see [Builds at SEEK] for more information. +4. [ ] Fill out [.me](.me) to power SEEK's system catalogue; + see the [Codex] documentation for more information. +5. [ ] Add Datadog configuration and data classification tags to [.gantry/common.yml](.gantry/common.yml); + see the [Gantry] documentation for more information. +6. [ ] Push local commits to the upstream GitHub branch. +7. [ ] Delete this checklist 😌. + ## Table of contents - [Design](#design) - [Development](#development) - [Test](#test) - [Lint](#lint) + - [Start](#start) - [Deploy](#deploy) - [Support](#support) - [Dev](#dev) @@ -15,7 +34,19 @@ ## Design -TODO: explain the design of your API here. +<%-repoName %> is a Node.js HTTP server built in line with our [technology strategy]. +It uses the [Express] middleware framework and common SEEK packages. +Resource APIs enable synchronous interactions and serve as the backbone of SEEK's general service architecture. + +This project is deployed as a containerised application with [Gantry]. +A typical resource API instance does not require more than 1 vCPU, +so we eschew clustering configurations in favour of a single Node.js process per container. +Under load, we autoscale horizontally in terms of container count up to `autoScaling.maxCount`. + +Gantry configures [CodeDeploy] for a blue-green deployment approach. +A smoke test is run against the new version before traffic is switched over, +providing an opportunity to test access and connectivity to online dependencies. +This defaults to an HTTP request to the `GET /smoke` endpoint. ## Development @@ -35,12 +66,27 @@ yarn format yarn lint ``` +### Start + +```shell +# Start a local HTTP server +yarn start + +# Start with Node.js Inspector enabled +yarn start:debug +``` + ### Deploy This project is deployed through a [Buildkite pipeline](.buildkite/pipeline.yml). -- Branch commits can be deployed to the dev environment by unblocking a step in the Buildkite UI -- Master commits are automatically deployed to the dev and prod environments in sequence +- Commits to a feature branch can be deployed to the dev environment by unblocking a step in the Buildkite UI +- Commits to the default branch are automatically deployed to the dev and prod environments in sequence + +To rapidly roll back a change, +retry an individual deployment step from the previous build in Buildkite. +Note that this will introduce drift between the head of the default Git branch and the live environment; +use with caution and always follow up with a proper revert or fix in Git history. ## Support @@ -63,3 +109,10 @@ TODO: add support links for the prod environment. - Datadog dashboard - Splunk logs --> + +[builds at seek]: https://builds-at-seek.ssod.skinfra.xyz +[codedeploy]: https://docs.aws.amazon.com/codedeploy +[codex]: https://codex.ssod.skinfra.xyz/docs +[express]: https://expressjs.com +[gantry]: https://gantry.ssod.skinfra.xyz +[technology strategy]: https://tech-strategy.ssod.skinfra.xyz diff --git a/template/greeter/README.md b/template/greeter/README.md index efa94b8c2..828bfb26c 100644 --- a/template/greeter/README.md +++ b/template/greeter/README.md @@ -2,29 +2,107 @@ [![Powered by skuba](https://img.shields.io/badge/🤿%20skuba-powered-009DC4)](https://github.com/seek-oss/skuba) - + +### Prod + +TODO: add support links for the prod environment. + + + +[builds at seek]: https://builds-at-seek.ssod.skinfra.xyz +[codex]: https://codex.ssod.skinfra.xyz/docs +[technology strategy]: https://tech-strategy.ssod.skinfra.xyz diff --git a/template/koa-rest-api/README.md b/template/koa-rest-api/README.md index 9fd2e548b..a1e918b9a 100644 --- a/template/koa-rest-api/README.md +++ b/template/koa-rest-api/README.md @@ -2,12 +2,31 @@ [![Powered by skuba](https://img.shields.io/badge/🤿%20skuba-powered-009DC4)](https://github.com/seek-oss/skuba) +Next steps: + +1. [ ] Finish templating if this was skipped earlier: + + ```shell + yarn skuba configure + ``` + +2. [ ] Create a new repository in the appropriate GitHub organisation. +3. [ ] Add the repository to BuildAgency; + see [Builds at SEEK] for more information. +4. [ ] Fill out [.me](.me) to power SEEK's system catalogue; + see the [Codex] documentation for more information. +5. [ ] Add Datadog configuration and data classification tags to [.gantry/common.yml](.gantry/common.yml); + see the [Gantry] documentation for more information. +6. [ ] Push local commits to the upstream GitHub branch. +7. [ ] Delete this checklist 😌. + ## Table of contents - [Design](#design) - [Development](#development) - [Test](#test) - [Lint](#lint) + - [Start](#start) - [Deploy](#deploy) - [Support](#support) - [Dev](#dev) @@ -15,7 +34,24 @@ ## Design -TODO: explain the design of your API here. +<%-repoName %> is a Node.js HTTP server built in line with our [technology strategy]. +It uses the [Koa] middleware framework and common SEEK packages. +Resource APIs enable synchronous interactions and serve as the backbone of SEEK's general service architecture. + +The `koa-rest-api` template is modelled after a hypothetical service for posting and retrieving job advertisements. +It's stubbed out with in-memory [storage](src/storage) which can observed by standing up an environment with multiple instances. +Storage is local to each instance, so load balancing across the instances may render a read inconsistent with a previous write. +This would be replaced with an external data store in production. + +This project is deployed as a containerised application with [Gantry]. +A typical resource API instance does not require more than 1 vCPU, +so we eskew clustering configurations in favour of a single Node.js process per container. +Under load, we autoscale horizontally in terms of container count up to `autoScaling.maxCount`. + +Gantry configures [CodeDeploy] for a blue-green deployment approach. +A smoke test is run against the new version before traffic is switched over, +providing an opportunity to test access and connectivity to online dependencies. +This defaults to an HTTP request to the `GET /smoke` endpoint. ## Development @@ -28,19 +64,34 @@ yarn test ### Lint ```shell -# fix +# Fix issues yarn format -# check +# Check for issues yarn lint ``` +### Start + +```shell +# Start a local HTTP server +yarn start + +# Start with Node.js Inspector enabled +yarn start:debug +``` + ### Deploy This project is deployed through a [Buildkite pipeline](.buildkite/pipeline.yml). -- Branch commits can be deployed to the dev environment by unblocking a step in the Buildkite UI -- Master commits are automatically deployed to the dev and prod environments in sequence +- Commits to a feature branch can be deployed to the dev environment by unblocking a step in the Buildkite UI +- Commits to the default branch are automatically deployed to the dev and prod environments in sequence + +To rapidly roll back a change, +retry an individual deployment step from the previous build in Buildkite. +Note that this will introduce drift between the head of the default Git branch and the live environment; +use with caution and always follow up with a proper revert or fix in Git history. ## Support @@ -63,3 +114,10 @@ TODO: add support links for the prod environment. - Datadog dashboard - Splunk logs --> + +[builds at seek]: https://builds-at-seek.ssod.skinfra.xyz +[codedeploy]: https://docs.aws.amazon.com/codedeploy +[codex]: https://codex.ssod.skinfra.xyz/docs +[gantry]: https://gantry.ssod.skinfra.xyz +[koa]: https://koajs.com +[technology strategy]: https://tech-strategy.ssod.skinfra.xyz diff --git a/template/lambda-sqs-worker/README.md b/template/lambda-sqs-worker/README.md index 0d432b2a5..458872618 100644 --- a/template/lambda-sqs-worker/README.md +++ b/template/lambda-sqs-worker/README.md @@ -2,12 +2,30 @@ [![Powered by skuba](https://img.shields.io/badge/🤿%20skuba-powered-009DC4)](https://github.com/seek-oss/skuba) +Next steps: + +1. [ ] Finish templating if this was skipped earlier: + + ```shell + yarn skuba configure + ``` + +2. [ ] Create a new repository in the appropriate GitHub organisation. +3. [ ] Add the repository to BuildAgency; + see [Builds at SEEK] for more information. +4. [ ] Fill out [.me](.me) to power SEEK's system catalogue; + see the [Codex] documentation for more information. +5. [ ] Add deployment bucket configuration and data classification tags to [serverless.yml](serverless.yml). +6. [ ] Push local commits to the upstream GitHub branch. +7. [ ] Delete this checklist 😌. + ## Table of contents - [Design](#design) - [Development](#development) - [Test](#test) - [Lint](#lint) + - [Start](#start) - [Deploy](#deploy) - [Support](#support) - [Dev](#dev) @@ -15,44 +33,87 @@ ## Design -TODO: explain the design of your worker here. +<%-repoName %> is a Node.js [Lambda] application built in line with our [technology strategy]. +It is backed by a typical SQS message + dead letter queue configuration and uses common SEEK packages. +Workers enable fault-tolerant asynchronous processing of events. + +The `lambda-sqs-worker` template is modelled after a hypothetical enricher that scores job advertisements. +It's stubbed out with in-memory [scoring service](src/services/jobScorer.ts). +This would be replaced with internal logic or an external service in production. + +This project is deployed with [Serverless]. +The Lambda runtime provisions a single Node.js process per container. +The supplied [serverless.yml](serverless.yml) starts out with a minimal `memorySize` which may require tuning based on workload. +Under load, we autoscale horizontally in terms of container count up to `reservedConcurrency`. + +Serverless configures [CodeDeploy] for a blue-green deployment approach. +A smoke test is run against the new version before traffic is switched over, +providing an opportunity to test access and connectivity to online dependencies. +This defaults to an invocation with an empty object `{}`, per [src/hooks.ts](src/hooks.ts). ## Development ### Test ```shell +# Run Jest tests locally yarn test + +# Authenticate to dev account +awsauth + +# Run smoke test against deployed application +ENVIRONMENT=dev yarn smoke ``` ### Lint ```shell -# fix +# Fix issues yarn format -# check +# Check for issues yarn lint ``` +### Start + +```shell +# Start a local HTTP server +yarn start + +# Start with Node.js Inspector enabled +yarn start:debug +``` + +This serves the Lambda application over HTTP. +For example, to invoke the handler with an empty object `{}` for smoke testing: + +```shell +curl --data '[{}, {"awsRequestId": "local"}]' --include localhost:<%- port %> +``` + ### Deploy This project is deployed through a [Buildkite pipeline](.buildkite/pipeline.yml). -- Branch commits can be deployed to the dev environment by unblocking a step in the Buildkite UI -- Master commits are automatically deployed to the dev and prod environments in sequence +- Commits to a feature branch can be deployed to the dev environment by unblocking a step in the Buildkite UI +- Commits to the default branch are automatically deployed to the dev and prod environments in sequence -To deploy from your local machine: +To deploy locally: ```shell -# authenticate to your dev account +# Authenticate to dev account awsauth -yarn build - ENVIRONMENT=dev yarn deploy ``` +To rapidly roll back a change, +retry an individual deployment step from the previous build in Buildkite. +Note that this will introduce drift between the head of the default Git branch and the live environment; +use with caution and always follow up with a proper revert or fix in Git history. + ## Support ### Dev @@ -74,3 +135,10 @@ TODO: add support links for the prod environment. - Datadog dashboard - Splunk logs --> + +[builds at seek]: https://builds-at-seek.ssod.skinfra.xyz +[codedeploy]: https://docs.aws.amazon.com/codedeploy +[codex]: https://codex.ssod.skinfra.xyz/docs +[lambda]: https://docs.aws.amazon.com/lambda +[serverless]: https://www.serverless.com/ +[technology strategy]: https://tech-strategy.ssod.skinfra.xyz diff --git a/template/oss-npm-package/README.md b/template/oss-npm-package/README.md index 1c3dd9a19..f3d9a5d45 100644 --- a/template/oss-npm-package/README.md +++ b/template/oss-npm-package/README.md @@ -6,13 +6,27 @@ This package is intended to be public on [seek-oss]. To create an internal package, run `skuba init` and select the `private-npm-package` template. -Please read [SEEK's Open Source RFC] before proceeding. +Next steps: + +1. [ ] Read [SEEK's Open Source RFC]. +2. [ ] Create a new repository in the [seek-oss] GitHub organisation. +3. [ ] Fill out [.me](.me) to power SEEK's system catalogue; + see the [Codex] documentation for more information. +4. [ ] Push local commits to the upstream GitHub branch. +5. [ ] Delete this checklist 😌. ## Table of contents - [API](#api) + - [`log`](#log) - [Development](#development) + - [Prerequisites](#prerequisites) + - [Test](#test) + - [Lint](#lint) + - [Package](#package) - [Release](#release) + - [Releasing latest](#releasing-latest) + - [Releasing other dist-tags](#releasing-other-dist-tags) ## API @@ -47,20 +61,20 @@ yarn test ### Lint ```shell -# fix +# Fix issues yarn format -# check +# Check for issues yarn lint ``` ### Package ```shell -# compile source +# Compile source yarn build -# review bundle +# Review bundle npm pack ``` @@ -107,6 +121,7 @@ Here are some branches that **semantic-release** supports by default: For more information, see the **semantic-release** docs on [triggering a release]. +[codex]: https://codex.ssod.skinfra.xyz/docs [distribution tags]: https://docs.npmjs.com/adding-dist-tags-to-packages [oss npm package guidance]: https://github.com/SEEK-Jobs/seek-oss-ci/blob/master/NPM_PACKAGES.md#access-to-publish-to-npm [release workflow]: .github/workflows/release.yml diff --git a/template/private-npm-package/README.md b/template/private-npm-package/README.md index 367f65379..22713de67 100644 --- a/template/private-npm-package/README.md +++ b/template/private-npm-package/README.md @@ -6,11 +6,27 @@ This package is intended to be private on [SEEK-Jobs] under the `@seek` npm org. To create a public package, run `skuba init` and select the `oss-npm-package` template. +Next steps: + +1. [ ] Create a new repository in the [SEEK-Jobs] GitHub organisation. +2. [ ] Follow [Gutenberg] instructions for [installing on your repository]. +3. [ ] Fill out [.me](.me) to power SEEK's system catalogue; + see the [Codex] documentation for more information. +4. [ ] Push local commits to the upstream GitHub branch. +5. [ ] Delete this checklist 😌. + ## Table of contents - [API](#api) + - [`log`](#log) - [Development](#development) + - [Prerequisites](#prerequisites) + - [Test](#test) + - [Lint](#lint) + - [Package](#package) - [Release](#release) + - [Releasing latest](#releasing-latest) + - [Releasing other dist-tags](#releasing-other-dist-tags) ## API @@ -45,20 +61,20 @@ yarn test ### Lint ```shell -# fix +# Fix issues yarn format -# check +# Check for issues yarn lint ``` ### Package ```shell -# compile source +# Compile source yarn build -# review bundle +# Review bundle npm pack ``` @@ -68,8 +84,6 @@ This package is published to the npm registry under the private `@seek` npm org It depends on this repo being hosted on [SEEK-Jobs] with appropriate access. -To set up this repo for publishing, follow the instructions in Gutenberg's [installing on your repository] guidance. - ### Releasing latest Commits to the `master` branch will be released with the `latest` tag, @@ -93,9 +107,10 @@ Here are some branches that **semantic-release** supports by default: For more information, see the **semantic-release** docs on [triggering a release]. +[codex]: https://codex.ssod.skinfra.xyz/docs [distribution tags]: https://docs.npmjs.com/adding-dist-tags-to-packages [gutenberg]: https://github.com/SEEK-Jobs/gutenberg [installing on your repository]: https://github.com/SEEK-Jobs/gutenberg#installing-on-your-repository -[seek-jobs]: https://github.com/SEEK-Jobs +[seek-jobs]: https://github.com/orgs/seek-jobs/sso [semantic-release]: https://github.com/semantic-release/semantic-release [triggering a release]: https://github.com/semantic-release/semantic-release/#triggering-a-release