Merge branch 'develop' into mm/fix-dtl-statedump-upload

boba_community/fraud-detector/Dockerfile.fraud-detector

@@ -0,0 +1,35 @@
+FROM omgx/builder AS builder
+FROM node:14-alpine
+
+RUN apk add --no-cache git curl python bash jq
+WORKDIR /opt/optimism/
+
+COPY --from=builder /optimism/*.json /optimism/yarn.lock ./
+COPY --from=builder /optimism/node_modules ./node_modules
+
+# copy deps (would have been nice if docker followed the symlinks required)
+COPY --from=builder /optimism/packages/core-utils/package.json ./packages/core-utils/package.json
+COPY --from=builder /optimism/packages/core-utils/dist ./packages/core-utils/dist
+COPY --from=builder /optimism/packages/common-ts/package.json ./packages/common-ts/package.json
+COPY --from=builder /optimism/packages/common-ts/dist ./packages/common-ts/dist
+COPY --from=builder /optimism/packages/hardhat-ovm/package.json ./packages/hardhat-ovm/package.json
+COPY --from=builder /optimism/packages/hardhat-ovm/dist ./packages/hardhat-ovm/dist
+
+# get the needed built artifacts
+WORKDIR /opt/optimism/packages/contracts
+COPY --from=builder /optimism/packages/contracts/dist ./dist
+COPY --from=builder /optimism/packages/contracts/*.json ./
+COPY --from=builder /optimism/packages/contracts/node_modules ./node_modules
+COPY --from=builder /optimism/packages/contracts/artifacts ./artifacts
+COPY --from=builder /optimism/packages/contracts/artifacts-ovm ./artifacts-ovm
+COPY --from=builder /optimism/packages/contracts/deployments ./deployments
+
+# copy the service
+WORKDIR /opt/optimism/boba_community/fraud-detector
+COPY dist ./dist
+COPY package.json .
+COPY exec ./exec
+COPY node_modules ./node_modules
+COPY scripts ./scripts
+
+ENTRYPOINT ["./scripts/fraud-detector.sh"]

boba_community/fraud-detector/README.md

@@ -0,0 +1,96 @@
+- [Fraud Detector](#fraud-detector)
+  * [0. Concepts](#0-concepts)
+  * [1. Errors and State Root Mismatches in the Boba Mainnet](#1-errors-and-state-root-mismatches-in-the-boba-mainnet)
+  * [2. What do when you discover a state root mismatch](#2-what-do-when-you-discover-a-state-root-mismatch)
+  * [3. Running the Fraud Detector, the Verifier, and the Data Transport Layer (DTL)](#3-running-the-fraud-detector--the-verifier--and-the-data-transport-layer--dtl-)
+
+# Fraud Detector
+
+A docker script for running a *Verifier*, a *DTL* (data transport layer), and a *fraud-detector* service.
+
+## 0. Concepts
+
+This repo allows you to: 
+
+1. Run your own Boba geth L2 on your computer. In this case, the geth L2 will run in its `Verifier` mode. In `Verifier` mode, the geth will sync from L1 and use the transaction data from the L1 contracts to compute what the state roots should be, *if the operator is honest*.
+
+2. A separate service, the *fraud-detector*, can then be used to discover potential fraud. Briefly, the fraud detection process consists of requesting a state root from Boba Mainnet L1 and requesting a state root from your Verifier. If those state roots match, then, the operator has been honest. If they do not match, then, that **might** be due to fraud, or, could also indicate indexing or timestamp errors, or chain configuration errors.
+
+The central idea is that if two (or more) systems look at the same transactions, then they should all generate the same state roots. If they don't, then there is a problem somewhere. Fundamentally, the security of rollups has little to do with math or cryptography - rather, security arises from the operator publicly depositing transactions and their corresponding state roots, and then, **having many independent nodes check those data for possible discrepancies**.
+
+## 1. Errors and State Root Mismatches in the Boba Mainnet
+
+* For the first 10 blocks, the chainID was set (incorrectly) to 28 rather than 288. Therefore, the EIP155 signatures fail for those blocks, and the Verifier cannot sync those blocks. This has been addressed by setting the L1_MAINNET_DEPLOYMENT_BLOCK to 10 blocks past the zero block. 
+
+* There is one state root mismatch at L2 block 155, arising from a two second discrepancy in a timestamp, that was ultimately caused by a too-small setting for the number of confirmations (DATA_TRANSPORT_LAYER__CONFIRMATIONS). This value was therefore increased to 4. The 2 second block 155 timestamp discrepancy has been addressed in a custom docker image (`omgx/data-transport-layer:rc1.0-surgery`). 
+
+## 2. What do when you discover a state root mismatch
+
+Congratulations! The security of the L2 depends on community monitoring of the operator's actions. If you have discovered a state root mismatch, please file a GitHub issue (https://github.com/omgnetwork/optimism/issues). We should have a good response / clarification for you quickly. In the future, with the Boba governance token, additional mechanisms will be released to incentivize and reward community monitoring of the Boba L2.  
+
+## 3. Running the Fraud Detector, the Verifier, and the Data Transport Layer (DTL)
+
+**Requirements**: you will need a command line and Docker.
+
+**Open a terminal window**. Create a `.env` file from the provided example (`env.example`) and paste in your Infura key. You can get a free Infura key at https://infura.io. Your `.env` should then look like this (except that you will be using your Infura key):
+
+```bash
+
+L1_NODE_WEB3_URL=https://mainnet.infura.io/v3/YOUR_INFURA_KEY
+ADDRESS_MANAGER_ADDRESS=0x8376ac6C3f73a25Dd994E0b0669ca7ee0C02F089
+L1_MAINNET_DEPLOYMENT_BLOCK=13011896
+
+```
+
+Then, start the Fraud Prover, Verifier, and DTL by:
+
+```bash
+
+$ ./up_local.sh
+
+```
+
+The L2 will spin up and begin to sync with the Boba L1. **NOTE: the sync process can take ~2 hours to complete**. During the sync process, you will see the Verifier gradually catch up with the Boba L2:
+
+```bash
+
+data_transport_layer_1  | {"level":30,"time":1632868364830,"method":"GET","url":"/eth/syncing?backend=l1","elapsed":0,"msg":"Served HTTP Request"}
+geth_l2_1               | INFO [09-28|22:32:44.831] Still syncing                            index=9 tip=2706
+data_transport_layer_1  | {"level":30,"time":1632868374830,"method":"GET","url":"/eth/syncing?backend=l1","elapsed":1,"msg":"Served HTTP Request"}
+geth_l2_1               | INFO [09-28|22:32:54.831] Still syncing                            index=11 tip=2706
+
+```
+
+When your Verifier has caught up with the Boba L2, then you will see it fetching transactions and performing other L2 operations: 
+
+```bash
+
+data_transport_layer_1  | {"level":30,"time":1632875212812,"method":"GET","url":"/batch/transaction/latest","elapsed":1,"msg":"Served HTTP Request"}
+geth_l2_1               | INFO [09-29|00:26:52.813] Set L2 Gas Price                         gasprice=0
+
+```
+
+The Fraud Detector will then fire up and cache relevant events from the chain. After caching older chain data, which should take at most 30 minutes, the Fraud Detector will then verify each state root: 
+
+```bash
+
+data_transport_layer_1  | {"level":30,"time":1632965735657,"method":"GET","url":"/eth/gasprice","elapsed":1310,"msg":"Served HTTP Request"}
+geth_l2_1               | INFO [09-30|01:35:35.658] Set L1 Gas Price                         gasprice=88121008566
+data_transport_layer_1  | {"level":30,"time":1632965735660,"method":"GET","url":"/batch/transaction/latest","elapsed":1,"msg":"Served HTTP Request"}
+geth_l2_1               | INFO [09-30|01:35:35.661] Set L2 Gas Price                         gasprice=3000000000
+fraud_detector_1        | New L1 blocks to inspect: 41
+fraud_detector_1        | Scanning L1 from 13324134 to 13324175
+fraud_detector_1        | Adding 0 new L2blocks: []
+geth_l2_1               | DEBUG[09-30|01:35:36.232] Served eth_getBlockByNumber              conn=172.18.0.4:40504 reqid=858 t=219.754µs
+fraud_detector_1        | {"level":30,"time":1632965736232,
+      "L2_block":825,
+      "operatorSR":"0x26ec701d6375df51b074c8e9efb1f07e7edcd1e8bcd10a2c356442db80a45fe1",
+      "verifierSR":"0x26ec701d6375df51b074c8e9efb1f07e7edcd1e8bcd10a2c356442db80a45fe1",
+      "msg":"State root MATCH - verified ✓"
+    }
+fraud_detector_1        | 
+fraud_detector_1        | ***********************************************************
+fraud_detector_1        | State root MATCH - verified ✓ L2 Block number 825
+fraud_detector_1        | ***********************************************************
+
+```

boba_community/fraud-detector/docker-detect.yml

@@ -0,0 +1,120 @@
+version: "3.4"
+
+x-var: &L1_NODE_WEB3_URL
+  L1_NODE_WEB3_URL=${L1_NODE_WEB3_URL}
+
+x-var: &DATA_TRANSPORT_LAYER__L1_RPC_ENDPOINT
+  DATA_TRANSPORT_LAYER__L1_RPC_ENDPOINT=${L1_NODE_WEB3_URL}
+
+x-var: &DEPLOYER_HTTP
+  DEPLOYER_HTTP=http://replica.boba.network:8081
+
+x-var: &ADDRESS_MANAGER_ADDRESS
+  ADDRESS_MANAGER_ADDRESS=${ADDRESS_MANAGER_ADDRESS}
+
+x-var: &DATA_TRANSPORT_LAYER__ADDRESS_MANAGER
+  DATA_TRANSPORT_LAYER__ADDRESS_MANAGER=${ADDRESS_MANAGER_ADDRESS}
+
+x-var: &L1_MAINNET_DEPLOYMENT_BLOCK
+  L1_MAINNET_DEPLOYMENT_BLOCK=${L1_MAINNET_DEPLOYMENT_BLOCK}
+
+x-var: &ETH1_CTC_DEPLOYMENT_HEIGHT
+  ETH1_CTC_DEPLOYMENT_HEIGHT=${L1_MAINNET_DEPLOYMENT_BLOCK}
+
+services:
+
+  data_transport_layer:
+    image: omgx/data-transport-layer:rc1.0-surgery
+    environment:
+      - *L1_NODE_WEB3_URL
+      - *DEPLOYER_HTTP
+      - *DATA_TRANSPORT_LAYER__L1_RPC_ENDPOINT
+      - *DATA_TRANSPORT_LAYER__ADDRESS_MANAGER
+      - DATA_TRANSPORT_LAYER__SYNC_FROM_L1=true
+      - DATA_TRANSPORT_LAYER__SYNC_FROM_L2=false
+      - DATA_TRANSPORT_LAYER__L2_RPC_ENDPOINT=https://replica.boba.network
+      - DATA_TRANSPORT_LAYER__L2_CHAIN_ID=288
+      - DATA_TRANSPORT_LAYER__DB_PATH=/db
+      - DATA_TRANSPORT_LAYER__SERVER_PORT=7878
+      - DATA_TRANSPORT_LAYER__TRANSACTIONS_PER_POLLING_INTERVAL=1000
+      - DATA_TRANSPORT_LAYER__CONFIRMATIONS=4
+      - DATA_TRANSPORT_LAYER__POLLING_INTERVAL=5000
+      - DATA_TRANSPORT_LAYER__LOGS_PER_POLLING_INTERVAL=2000
+      - DATA_TRANSPORT_LAYER__DANGEROUSLY_CATCH_ALL_ERRORS=true
+      - DATA_TRANSPORT_LAYER__SERVER_HOSTNAME=0.0.0.0
+      - URL=http://replica.boba.network:8081/addresses.json
+      - RETRIES=1000
+    restart: always
+    logging:
+      driver: "json-file"
+      options:
+        max-file: "5"
+        max-size: "10m"
+    ports:
+      - 7879:7878
+
+  geth_l2:
+    image: omgx/l2geth:rc1.0
+    environment:
+      - *DEPLOYER_HTTP
+      - *L1_NODE_WEB3_URL
+      - *ETH1_CTC_DEPLOYMENT_HEIGHT
+      - ETH1_SYNC_SERVICE_ENABLE=true
+      - ETH1_CONFIRMATION_DEPTH=0
+      - ROLLUP_CLIENT_HTTP=http://data_transport_layer:7878
+      - ROLLUP_POLL_INTERVAL_FLAG=3s
+      - USING_OVM=true
+      - CHAIN_ID=288
+      - NETWORK_ID=288
+      - DEV=true
+      - DATADIR=/root/.ethereum
+      - RPC_ENABLE=true
+      - RPC_ADDR=geth_l2
+      - RPC_CORS_DOMAIN=*
+      - RPC_VHOSTS=*
+      - RPC_PORT=8545
+      - WS=true
+      - WS_ADDR=0.0.0.0
+      - IPC_DISABLE=true
+      - TARGET_GAS_LIMIT=11000000
+      - RPC_API=eth,net,rollup,web3,txpool,debug,db,personal
+      - WS_API=eth,net,rollup,web3,txpool,debug,db,personal
+      - WS_ORIGINS=*
+      - GASPRICE=0
+      - NO_USB=true
+      - GCMODE=archive
+      - NO_DISCOVER=true
+      - ROLLUP_STATE_DUMP_PATH=https://replica.boba.network:8081/state-dump.latest.json
+      - URL=https://replica.boba.network:8081/addresses.json
+      - ROLLUP_ENABLE_L2_GAS_POLLING=true
+      - ROLLUP_ENFORCE_FEES=true
+      - ETH1_L1_FEE_WALLET_ADDRESS=0xbF159Ba5E5917551c70E377ADf21eD3736209fCE
+      - ROLLUP_GAS_PRICE_ORACLE_OWNER_ADDRESS=0xd86D22c02E301BE7C35e3Ef20962f614cAf32B76
+      - ROLLUP_BACKEND=l1
+      - ROLLUP_VERIFIER_ENABLE=true
+      - RETRIES=1000
+    restart: always
+    entrypoint: sh ./geth.sh
+    logging:
+      driver: "json-file"
+      options:
+        max-file: "5"
+        max-size: "10m"
+    ports:
+      - 8547:8545
+      - 8548:8546
+
+  fraud_detector:
+    image: omgx/fraud-detector:latest
+    environment:
+      - *L1_NODE_WEB3_URL
+      - *ADDRESS_MANAGER_ADDRESS
+      - *DEPLOYER_HTTP
+      - *L1_MAINNET_DEPLOYMENT_BLOCK
+      - VERIFIER_WEB3_URL=http://geth_l2:8545
+    restart: always
+    logging:
+      driver: "json-file"
+      options:
+        max-file: "5"
+        max-size: "10m"

boba_community/fraud-detector/env.example

@@ -0,0 +1,3 @@
+L1_NODE_WEB3_URL=https://mainnet.infura.io/v3/YOUR_INFURA_KEY
+ADDRESS_MANAGER_ADDRESS=0x8376ac6C3f73a25Dd994E0b0669ca7ee0C02F089
+L1_MAINNET_DEPLOYMENT_BLOCK=13011896

boba_community/fraud-detector/exec/run-fraud-detector.js

@@ -0,0 +1,10 @@
+#!/usr/bin/env node
+
+const main = require("../dist/exec/run").default
+
+;(async () => {
+  await main()
+})().catch((err) => {
+  console.log(err)
+  process.exit(1)
+})

boba_community/fraud-detector/package.json

@@ -0,0 +1,50 @@
+{
+  "name": "@boba/fraud-detector",
+  "version": "0.0.4",
+  "description": "Fraud Detector Service",
+  "main": "dist/index",
+  "types": "dist/index",
+  "files": [
+    "dist/index"
+  ],
+  "scripts": {
+    "start": "node ./exec/run-fraud-detector.js",
+    "build": "tsc -p ./tsconfig.build.json"
+  },
+  "keywords": [
+    "optimism",
+    "ethereum",
+    "boba",
+    "omgnetwork",
+    "fraud-detector"
+  ],
+  "homepage": "https://github.com/omgnetwork/optimism/blob/develop/boba_community/fraud-detector#readme",
+  "license": "MIT",
+  "author": "OMG/Boba",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/omgnetwork/optimism.git"
+  },
+  "dependencies": {
+    "@eth-optimism/common-ts": "^0.1.5",
+    "@eth-optimism/contracts": "^0.4.12",
+    "@eth-optimism/core-utils": "^0.5.5",
+    "bcfg": "^0.1.6",
+    "chalk": "^4.1.2",
+    "dotenv": "^8.2.0",
+    "merkle-patricia-tree": "4.0.0",
+    "merkletreejs": "^0.2.10",
+    "rlp": "^2.2.6",
+    "sqlite3": "^5.0.2",
+    "typescript": "^4.3.2"
+  },
+  "devDependencies": {
+    "@ethersproject/abstract-provider": "^5.0.9",
+    "@ethersproject/providers": "^5.0.21",
+    "@nomiclabs/hardhat-ethers": "^2.0.2",
+    "chai": "^4.3.0",
+    "ethereumjs-util": "^7.0.5",
+    "ethers": "^5.0.31",
+    "mocha": "^8.3.0"
+  }
+}

boba_community/fraud-detector/scripts/fraud-detector.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+
+set -e
+
+RETRIES=${RETRIES:-1000}
+
+until $(curl --silent --fail \
+    --output /dev/null \
+    -H "Content-Type: application/json" \
+    --data "$JSON" "$VERIFIER_WEB3_URL"); do
+  sleep 5
+  echo "Will wait $((RETRIES--)) more times for $VERIFIER_WEB3_URL to be up and fully synced..."
+
+  if [ "$RETRIES" -lt 0 ]; then
+    echo "Timeout waiting for verifier at $VERIFIER_WEB3_URL"
+    exit 1
+  fi
+done
+echo "Connected to Verifier at $VERIFIER_WEB3_URL"
+
+# go
+exec node ./exec/run-fraud-detector.js