Adding environment creation / maintenance instructions

aws/README.md

@@ -0,0 +1,93 @@
+# AWS
+This documentation contains all of the necessary information to build and deploy new images to an existing environment or to create an entirely new environment in AWS.
+
+# Building and Registering Containers
+There are a few different docker containers in this repo that get built and published to AWS's registry. 
+As of right now they are defined by the Dockerfiles at:
+* `docker/geth/Dockerfile` published (here)
+  * Manually built and deployed
+* `Dockerfile`
+  * Automatically built and deployed to shared dev environment on merge to `master`
+  * Note: 
+    * This is for the rollup-full-node but at the base directory for easy dependency inclusion
+    * Currently all source code must be built _outside_ of Docker, ahead of doing a docker build.
+
+Here are the steps for publishing new version to ECR:
+* `docker/geth/Dockerfile` click on "View push commands" [here](https://us-east-2.console.aws.amazon.com/ecr/repositories/optimism/geth/?region=us-east-2) 
+* `Dockerfile` click on "View push commands" [here](https://us-east-2.console.aws.amazon.com/ecr/repositories/optimism/rollup-full-node/?region=us-east-2) 
+
+
+# Creating an AWS ECS Environment
+The contents of this directory can be used to deploy a fully-functional Full Node to AWS.
+
+Below are some instructions on how to do so. For more info, the instructions below loosely follow [this tutorial](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-cli-tutorial-ec2.html).
+
+## Prerequisites
+AWS:
+* Set up an AWS Account, Access Key & Secret, and keypair
+* Install the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
+* Install the [AWS ECS CLI](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_CLI_installation.html)
+
+Other:
+* Install [Docker](https://docs.docker.com/docker-for-mac/install/)
+## Steps
+
+### 1) Configure the Amazon ECS CLI
+1. Create a cluster configuration:
+    ```
+    ecs-cli configure --cluster dev-full-node --default-launch-type EC2 --config-name dev-full-node-config --region us-east-2
+    ```
+    Note: choose an appropriate cluster name and have the config name derived from it.
+
+2. Create a profile to use to create the environment
+    ```
+    ecs-cli configure profile --access-key <your access key here> --secret-key <your secret here> --profile-name dev-full-node-profile
+    ```
+
+### 2) Create the Cluster
+```
+ecs-cli up --keypair <your keypair name> --capability-iam --size 1 --instance-type t3.micro --cluster-config dev-full-node-config --ecs-profile dev-full-node-profile --port 8545 --security-group <Security Group ID for the instance> --vpc <VPC ID for the instance> --subnets <comma-separated list of subnet IDs>
+```
+Note:
+* `size` is the number of ECS instances to create
+* `instance-type` can be bumped up for a more powerful environment. More info [here](https://aws.amazon.com/ec2/instance-types/)
+* If you haven't already, you'll need to create a Security Group. This is not individual to you, so see if your organization already has a suitable one.
+
+
+This may take a few minutes to finish. The result will be a fully provisioned EC2 instance on which your service/task will be deployed.
+
+### 3) Choose the appropriate `docker-compose.yml` and `ecs-params.yml`
+For the rest of the commands, you'll need to be in a directory with a `docker-compose.yml` and an `ecs-params.yml`. These will define the service(s) you are going to create. Change to the appropriate directory or create one of your own basing them off of an existing one.
+
+### 4) Deploy Tasks to Cluster 
+```
+ecs-cli compose up --create-log-groups --cluster-config dev-full-node-config --ecs-profile dev-full-node-profile
+```
+
+This will just start up the task(s) under the appropriate cluster. Ultimately we want a service to manage our task(s), but we don't want to do that until we know our tasks work. Make sure your tasks are functioning properly by checking their status and possibly even logging into CloudWatch and looking at the logs.
+
+To check the status of your task(s), run:
+```
+ecs-cli ps --cluster-config dev-full-node-config --ecs-profile dev-full-node-profile
+```
+
+### 5) Create the ECS Service
+First, kill the PoC tasks:
+```
+ecs-cli compose down --cluster-config dev-full-node-config --ecs-profile dev-full-node-profile
+```
+
+Now create the service:
+```
+ecs-cli compose service up --cluster-config dev-full-node-config --ecs-profile dev-full-node-profile
+```
+
+## Volumes
+Right now volumes needed by the various containers are configured to be locally stored on the EC2 instance on which the containers are run and automatically created if not present. Eventually we will want to move to a more redundant form of storage (like EBS mounts), but this is fine for now.
+
+For now, if you want to modify/delete the data in an environment you will need to
+* `ssh` into the EC2 instance running the containers
+* Find the volume(s) you would like to modify/delete (they are located at `/var/lib/docker/volumes`)
+* Modify/Delete them as necessary
+  * IMPORTANT: This will likely mess up any running tasks, so make sure to kill the task before
+  * Also note that the Service will auto-replace your tasks, so if you can't do this quickly, disable that feature during your maintenance.  

aws/dev/full-node/docker-compose.yml

@@ -0,0 +1,63 @@
+version: "3"
+services:
+  rollup-full-node:
+    image: <aws_account_id>.dkr.ecr.us-east-2.amazonaws.com/optimism/rollup-full-node:latest
+    depends_on: [ "geth_l2" ]
+    volumes:
+      - full-node-data:/mnt/full-node:rw
+      - l1-node-data:/mnt/l1-node:rw
+      - l2-node-data:/mnt/l2-node:rw
+    ports:
+      - 8545:8545
+    environment:
+      - OPCODE_WHITELIST_MASK
+      - L1_SEQUENCER_MNEMONIC
+      - L2_TO_L1_MESSAGE_RECEIVER_ADDRESS
+      - L2_TO_L1_MESSAGE_FINALITY_DELAY_IN_BLOCKS
+      - L2_RPC_SERVER_HOST
+      - L2_RPC_SERVER_PORT
+      - L2_WALLET_MNEMONIC
+      - L2_WALLET_PRIVATE_KEY_PATH=/mnt/l2-node/private_key.txt
+      - LOCAL_L1_NODE_PORT
+      - LOCAL_L1_NODE_PERSISTENT_DB_PATH=/mnt/l1-node
+      - L2_NODE_WEB3_URL=http://0.0.0.0:9545/
+
+    logging:
+      driver: awslogs
+      options:
+        awslogs-group: rollup-full-node
+        awslogs-region: us-east-2
+        awslogs-stream-prefix: l2-rpc-server
+
+  geth_l2:
+    image: <aws_account_id>.dkr.ecr.us-east-2.amazonaws.com/optimism/geth:latest
+    volumes:
+      - l2-node-data:/mnt/l2-node/l2:rw
+    environment:
+      - VOLUME_PATH=/mnt/l2-node/l2
+      - HOSTNAME=0.0.0.0
+      - PORT=9545
+      - NETWORK_ID=108
+      - KEYSTORE_PATH_SUFFIX=/keystore
+      - SEALER_PRIVATE_KEY_PATH_SUFFIX=/sealer_private_key.txt
+      - PRIVATE_KEY_PATH_SUFFIX=/private_key.txt
+      - ADDRESS_PATH_SUFFIX=/address.txt
+      - SEALER_ADDRESS_PATH_SUFFIX=/sealer_address.txt
+      - INITIAL_BALANCE=0x200000000000000000000000000000000000000000000000000000000000000
+      - GENISIS_PATH=etc/rollup-fullnode.json
+      - SETUP_RUN_PATH_SUFFIX=/setup_run.txt
+    ports:
+      - 9545:9545
+
+    logging:
+      driver: awslogs
+      options:
+        awslogs-group: rollup-full-node
+        awslogs-region: us-east-2
+        awslogs-stream-prefix: l2-node
+
+volumes:
+  full-node-data:
+  l1-node-data:
+  l2-node-data:
+

aws/dev/full-node/ecs-params.yml

@@ -0,0 +1,25 @@
+version: 1
+task_definition:
+  services:
+    full-node:
+      cpu_shares: 25
+      mem_limit: 524288000
+    geth_l2:
+      cpu_shares: 75
+      mem_limit: 1523288000
+  # This is all local for now -- eventually will change
+  ecs_network_mode: host
+  docker_volumes:
+      - name: l1-node-data
+        scope: shared
+        autoprovision: true
+        driver: 'local'
+      - name: l2-node-data
+        scope: shared
+        autoprovision: true       
+        driver: 'local'
+      - name: full-node-data
+        scope: shared
+        autoprovision: true       
+        driver: 'local'
+