feat: State Diff data for fraud proofs

[gakonst]

Fixes https://github.com/ethereum-optimism/roadmap/issues/209

Changes

• Adds a GetStateDiffProof API endpoint which takes in a block number and returns all the AccountResult which occurred in that block with their storageProof keys populated for all keys which were touched in that block
• Adds a StateDiffDb which provides APIs to write to a mapping of block => address => key[] and to read the state diff proofs for a block
• The keys per address per block are populated via the diffdb.SetDiffKey call made in ovm_statemanager.go::putcontractStorage. Should it be called in any other cases?

New JSON-RPC APIs:

• eth_getStateDiff
• eth_getStateDiffProof

TODO

• Convert DiffDB from a mapping to a sqlite3 instance
• Expose the new API
• DiffDB unit tests
• OVM State Manager unit tests
• E2E test with eth_getStateDiffProof`
• Add mutated to state diff proofs

E2E Testing

Compiling

I used the contract below:

pragma solidity ^0.5.16;
contract C {
    uint public x;

    constructor() public {  }

    function set(bool skip) public {
        if (skip) {
            x = x;
        } else {
            x = 2 * (x  + 1);
        }
    }
}

• It seems like opti-solc@develop-0.5 does not produce correct bytecode, as I wrote in the Slack.
• Using npm i -g @eth-optimism/solc@0.5.16-alpha.6 worked
• solcjs --bin --abi C.sol -> produces C_sol_C.abi and C_sol_C.bin

Runing geth

make geth
USING_OVM=true ./build/bin/geth --rpc --dev --rpcaddr "127.0.0.1" --rpccorsdomain "*" --networkid 420 --rpcapi 'eth,net,debug,web3' --gasprice '0' --targetgaslimit 12000000 --nousb --gcmode=archive --verbosity 5      

Testing

A dirty script for testing can be seen below. It deploys a simple contract which has been deployed with the aforementioned procedure, calls its methods, and then checks the state diff proof value.

const ethers = require('ethers')
const fs = require('fs')
const ABI = JSON.parse(fs.readFileSync("./C_sol_C.abi"))
const BIN = fs.readFileSync("./C_sol_C.bin", 'utf-8')

;(async () => {
    const provider = new ethers.providers.JsonRpcProvider("http://localhost:8545")
    const signer = new ethers.Wallet("0xd6c5c94c0ee962f86b9dc5bfb026e161ee1d311f951db848f2a5e7f5cfa21c59", provider)
    // const signer = new ethers.Wallet("0xa5c08c041c2441e2cdc48e0bcdf92f10ba7f84920cef7bd915a85a4f374b1914", provider)
    console.log(signer.address)

    let contract
    if (process.env.ADDRESS) {
        contract = new ethers.Contract(process.env.ADDRESS, ABI, signer)
    } else {
        const factory = new ethers.ContractFactory(ABI, BIN, signer)
        console.log("deploying..")
        contract = await factory.deploy()
        const receipt = await contract.deployTransaction.wait()
        console.log('deployed', receipt.contractAddress)
    }

    const go = async (skip) => {
        console.log("x", await contract.functions.x())
        console.log("setting..")
        await (await contract.set(skip)).wait()
        console.log("x", await contract.x())
        const blk = await provider.getBlockNumber()
        console.log("Getting proofs for", '0x' + blk.toString(16), blk.toString())
        const proofs = await provider.send("eth_getStateDiffProof", ['0x' + blk.toString(16)])
        console.log(JSON.stringify(proofs, null, 4))
    }

    await go(false)
    await go(true)
})()

The first logged JSON will set the contract's 0th storage slot to 0x2 with mutated = true, while the second one will again show that the 0th storage slot was touched, but mutated = false. Example output below:

First set call:

{
    "header": {
        "number": 2,
        "hash": "0x848bb720b100294e10c24757013b59983d0277cf41d8bfa0bc162eb9f120e217",
        "stateRoot": "0xff1845765de66f3d4132f1ce43f06b942e13e4c132053e755a757a71365dab45",
        "timestamp": 1606472840
    },
    "accounts": [
        {
            "address": "0x06a506a506a506a506a506a506a506a506a506a5",
            "accountProof": [
                "0xf843a0450004ac6003a1b8160a77783350e2f24623b4ac86a4974a33f17a94b3c2b0a9a03075a5cfec7f6f8cc16a11b2050ffcd2580d88287b80c84a3bafc31efe1b2f6480",
                "0xf843a0311654fcc79e3a3ce6b000388f6de838a9818bbf021b36bba255326f41450943a012b47eeb678560242e9fd3ae815e298111750829db23f62095bda91a41ac75f380",
                "0xf843a0481de25c6294e9660b7b9ed3d17508ed494c9d0a1652ff2bb6b0b928e1c2639aa052e16b762edb129a32e79c8aa0dd3c50548f5a2c18532bc0367dcfba0989579280",
                "0xf843a01c8f6cd76c623ed4309abb78c7a1a95a58d3087fe3d8d2adc90ff9bf80f8ba46a05c19662ed47e9d6d728259f87d926d4ce8452d9cad4b62bc9674af6197d5eace80",
                "0xf843a0fe0f143fbb448f8a65130b95d2b9a3b98271579112e657ce1a7b8ac119598a4ba012e2d121a725cbcdb96583916c702eda0e2b4b5e65fcd6499c736530dd041ca280",
                "0xf843a06bd7c76438c3e29cd1b81ab1ce41e4d94acb3dd201cfc890f7546c526eb0bfd4a04c8add91ebf424e3decd40b795c187fbef3e48a8d379ea8bd71a562d7c73f20780",
                "0xf843a082f6a3952de357e1d07c96cfbb32158226d3e80503c8261f62aefd236b0ec6a1a010eb1ca5d17a8f581938cd2e2ffdaec92d5b7116ad02a0374bde5566f6e5150c80",
                "0xf843a0bc4fd3af4941b398c74f929cd2c7fe4f8a1d865a6bcaa156a293efb9eec5388aa00c452cdea5e0638ab972a36a42c86acbad116c9c3955e1dcac1214eb6431504d80",
                "0xf843a0e67681156040e24a05c13c459b75c571de45fde155225ed003dc45aa68570c9da0262e28e76511f551b31bd3e56b9e53f6549301d9abda195bedfb1465cf4a3bef80",
                "0xf843a0ccb97d63e22d71575085ce5c806a2239d5fa087c1041f580fe0e55912178b67aa01fe864e07502c7c5ad5690fa68bb9a091d82270b7537f9d41a6d3f3da6deb43a80",
                "0xf843a0f7d390d9b706dd2cb016530b1ff4519a9bbb7ef79ebe1ce31013dfd88c5b2b30a0e05e5e0b253ea1e0e7308960410df6e36d96927cbec3019189d9cc2629e3078480",
                "0xf869a0ff9321e6643fc0c0ae270782a131a3eea1e12c916a16e4ee8c92e2a6e5444280b846f8441780a0017afcd532bfec2c8c628eae4b3c8847ab913b79f4dc573156bb160b8459b9d4a08207658eba4bb84c529dd4e02c7dee2c4ebcb37f1630d733a28dd91129182c79"
            ],
            "balance": "0x0",
            "codeHash": "0xc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470",
            "nonce": "0x0",
            "storageHash": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
            "storageProof": [
                {
                    "key": "0x0000000000000000000000000000000000000000000000000000000000000000",
                    "value": "0x0",
                    "proof": [],
                    "mutated": true
                },
                {
                    "key": "0x0000000000000000000000000000000000000000000000000000000000000003",
                    "value": "0x0",
                    "proof": [],
                    "mutated": false
                },
                {
                    "key": "0x0000000000000000000000000000000000000000000000000000000000000004",
                    "value": "0x0",
                    "proof": [],
                    "mutated": false
                },
                {
                    "key": "0x0000000000000000000000000000000000000000000000000000000000000001",
                    "value": "0x0",
                    "proof": [],
                    "mutated": true
                }
            ]
        },
        {
            "address": "0x4200000000000000000000000000000000000006",
            "accountProof": [
                "0xf843a0450004ac6003a1b8160a77783350e2f24623b4ac86a4974a33f17a94b3c2b0a9a03075a5cfec7f6f8cc16a11b2050ffcd2580d88287b80c84a3bafc31efe1b2f6480",
                "0xf843a0351a139c15a6063a4c2d83739a4886e8294d84a1cd961153512465cd71cacceda033ff935eb525fdec90600a978f91707d4e252f1cbf58d9d8a82eccc64326f54280",
                "0xf843a0c64184437278ab7917da8f734c25020b99cf87e1bc0eea2c4388f36fbb96c664a03f5aa0221bcaf63666dbeeb15ceb3f7212e09378245438a467ddec172500fd2380",
                "0xf843a0a4206ccb8a5b649d7b7740d35d699032f6f69359b086ebff43599b7bddb56e8aa0c6801d723545a08196db42f06ed7564e121f630595482d7f5deebd296f6b7abb80",
                "0xf843a0b9725113d3d912a79d1d4a94a54b64bf073cc54c52e92f732e2e3ae47b8c36dca03321c43b551c694d8ee608535ad07dc7fa8512f46f00fb8577a9141c0199798c80",
                "0xf843a0f35ee5ac004d6939689983c9128dc711f56a36279d2c879762b2e773099ad525a028e14e65cbc7ad582ddacc154cc0d04c51b20af71fe61dbb036f340d407cb55280",
                "0xf843a030d001d6cc9ca46ad62705f6db8b5d79629e3e52f2396ae57224cfb4a2988bb0a0e728d18846e394a5e5ff57c6480f67f549cf2cb9df3c083b48b8dec644e7404d80",
                "0xf843a057e8e3bdba1912bd574db05e9f0789e301ba9fafe5e937a1f86ef1736763eca8a0423c86bf5f6918f3820bb01e0dcd0cd3c119c1e1b92b02ed08139b974b6969fe80",
                "0xf843a023071354bba61ab96833c52286925bb5a8e603582743db51aebc55f86a575f19a034f2cfe18f5912f6bb431f2ffae321221f748e1937f1b1d2a206c8a8b73cee1080",
                "0xf843a0fff9290790d9f03d2081b4148379fe294ab04a73f8778d496938cfa09c9c74a2a07d179f55403c1066715312a618b7520ae4cff54af4f3bb0b1f103cbe5789e84080",
                "0xf843a03515e7be872ac031a314da4b13472ac17287a3c36b799495081f6f43fc6f7eaca0ba4ad8f350d65473e95c054a464f5a6e401554b30df00cfa12466efb133ac57980",
                "0xf869a0f93daad7526521f49e4a267f90a72e3b1871806fd705cd693c19b490d5251000b846f8448080a056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421a03d51cb4c14a5032f628f5e6cd20aebea0c2cb173617c3b2ef013692eb8ee47a3"
            ],
            "balance": "0x0",
            "codeHash": "0x3d51cb4c14a5032f628f5e6cd20aebea0c2cb173617c3b2ef013692eb8ee47a3",
            "nonce": "0x0",
            "storageHash": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
            "storageProof": [
                {
                    "key": "0xd4435d69fb8dfa20243856f7d55a4f6f87499860ebef0442d482d29b9b840a5b",
                    "value": "0x0",
                    "proof": [],
                    "mutated": false
                },
                {
                    "key": "0x8632b3478ce27e6c2251f16f71bf134373ff9d23cff5b8d5f95475fa6e52fe22",
                    "value": "0x0",
                    "proof": [],
                    "mutated": false
                }
            ]
        },
        {
            "address": "0xbae1ecd5647aca56326bfef7accc6720750f5a22",
            "accountProof": [
                "0xf843a0450004ac6003a1b8160a77783350e2f24623b4ac86a4974a33f17a94b3c2b0a9a03075a5cfec7f6f8cc16a11b2050ffcd2580d88287b80c84a3bafc31efe1b2f6480",
                "0xf843a0351a139c15a6063a4c2d83739a4886e8294d84a1cd961153512465cd71cacceda033ff935eb525fdec90600a978f91707d4e252f1cbf58d9d8a82eccc64326f54280",
                "0xf843a0c64184437278ab7917da8f734c25020b99cf87e1bc0eea2c4388f36fbb96c664a03f5aa0221bcaf63666dbeeb15ceb3f7212e09378245438a467ddec172500fd2380",
                "0xf843a0eef15e80e21d673fa86cf835d3086ce66518150e00ce8dbd4b00bb73a0e24132a0b71d6e6c4b4fbfdd67b640039d33b8d8025ffe25ca222b0fa26de8d08a61ba3580",
                "0xf843a0a72fbd949f87114db7370357fdf85a1ccb132d2a78bec4a421db8ad145c7ba0da02ede11c472c48937628287cb5bfd01675713cf0dc57c77acbb3fadede5c3607380",
                "0xf843a0ad6ccd10f054ca478bec2bad935646685492c019b2635e81efae04b34b4da1dca07b99f28c8f10b1e5c2ab9a767fae70b6d69e4ffbd7c7a430c52c9598dceef9dc80",
                "0xf843a05987be8f2d07c84eda8d97ff62be3da0423f5d3121575c34e6a14b8ced7d8f1aa01dfef3c94c73255bc56f102c86f2555770ab6a0aeb6e77c5af26b4fc153fd05f80",
                "0xf843a0e02b1276e6403d96d67980703fd52e377f16c94c3a1904e71a5404a785e9fbafa0d0cc63a77623c09bdd55ff3e496850ea79fd1f42a42e6f540724d6b18aaa676780",
                "0xe230a0609053ef433e1d35b2742ae5e3600652ec224d0cc57d7576214c371d164e501e",
                "0xf843a085369b0f0be366a6be3c7bb5c4bd6fec84d354db72de7f5ff892f7f037fb2555a05e43274e5e9a34e277e6dcffd0cbd2bf83b15482772295c38a4c0f3cf8f7b9c980",
                "0xf843a0852193fef48f6c8b61469868550143484cbed1c7efa40c650bfa924f21a5ae0da00551f886731fdb818bd9f6b06f6d4a3049f3dbf102c046bb1010c5cb37f087db80",
                "0xf869a0f7d7e771c8cf799a15567dae1a7625cb6deff893f6d3b73a09ba8714f4f9ce80b846f8448080a0d266be41e505c4aa98b1bd82cd7676571193ed7e571658e8a40da1897777efa4a0a799aac89d77d5783ca00ab6591670a05646eb19d18e0dc32f9d42d4d08ad19b"
            ],
            "balance": "0x0",
            "codeHash": "0xa799aac89d77d5783ca00ab6591670a05646eb19d18e0dc32f9d42d4d08ad19b",
            "nonce": "0x0",
            "storageHash": "0xd266be41e505c4aa98b1bd82cd7676571193ed7e571658e8a40da1897777efa4",
            "storageProof": [
                {
                    "key": "0x0000000000000000000000000000000000000000000000000000000000000000",
                    "value": "0x2",
                    "proof": [
                        "0xe3a1c290decd9548b62a8d60345a988386fc84ba6bc95484008f6362f93160ef3e563002"
                    ],
                    "mutated": true
                }
            ]
        }
    ]
}

Second call (does not modify):

{
    "header": {
        "number": 3,
        "hash": "0x037f863bddaabd7e0138b66ab36c2851ba7a1b5a82d5701c7fdac049f106e1e1",
        "stateRoot": "0x87b907a82be9717c38eee3ca034f6627e9f5599eb7ebbe6a3117d4a4884883a2",
        "timestamp": 1606472840
    },
    "accounts": [
        {
            "address": "0x06a506a506a506a506a506a506a506a506a506a5",
            "accountProof": [
                "0xf843a0e881ac781cfca939c599937dce13fc740c981a83c94c2a6f0019a5ff1eacc97ba03075a5cfec7f6f8cc16a11b2050ffcd2580d88287b80c84a3bafc31efe1b2f6480",
                "0xf843a0311654fcc79e3a3ce6b000388f6de838a9818bbf021b36bba255326f41450943a012b47eeb678560242e9fd3ae815e298111750829db23f62095bda91a41ac75f380",
                "0xf843a0481de25c6294e9660b7b9ed3d17508ed494c9d0a1652ff2bb6b0b928e1c2639aa052e16b762edb129a32e79c8aa0dd3c50548f5a2c18532bc0367dcfba0989579280",
                "0xf843a01c8f6cd76c623ed4309abb78c7a1a95a58d3087fe3d8d2adc90ff9bf80f8ba46a05c19662ed47e9d6d728259f87d926d4ce8452d9cad4b62bc9674af6197d5eace80",
                "0xf843a0fe0f143fbb448f8a65130b95d2b9a3b98271579112e657ce1a7b8ac119598a4ba012e2d121a725cbcdb96583916c702eda0e2b4b5e65fcd6499c736530dd041ca280",
                "0xf843a06bd7c76438c3e29cd1b81ab1ce41e4d94acb3dd201cfc890f7546c526eb0bfd4a04c8add91ebf424e3decd40b795c187fbef3e48a8d379ea8bd71a562d7c73f20780",
                "0xf843a082f6a3952de357e1d07c96cfbb32158226d3e80503c8261f62aefd236b0ec6a1a010eb1ca5d17a8f581938cd2e2ffdaec92d5b7116ad02a0374bde5566f6e5150c80",
                "0xf843a0bc4fd3af4941b398c74f929cd2c7fe4f8a1d865a6bcaa156a293efb9eec5388aa00c452cdea5e0638ab972a36a42c86acbad116c9c3955e1dcac1214eb6431504d80",
                "0xf843a0e67681156040e24a05c13c459b75c571de45fde155225ed003dc45aa68570c9da0262e28e76511f551b31bd3e56b9e53f6549301d9abda195bedfb1465cf4a3bef80",
                "0xf843a0ccb97d63e22d71575085ce5c806a2239d5fa087c1041f580fe0e55912178b67aa01fe864e07502c7c5ad5690fa68bb9a091d82270b7537f9d41a6d3f3da6deb43a80",
                "0xf843a0f7d390d9b706dd2cb016530b1ff4519a9bbb7ef79ebe1ce31013dfd88c5b2b30a0e05e5e0b253ea1e0e7308960410df6e36d96927cbec3019189d9cc2629e3078480",
                "0xf869a0ff9321e6643fc0c0ae270782a131a3eea1e12c916a16e4ee8c92e2a6e5444280b846f8441780a0017afcd532bfec2c8c628eae4b3c8847ab913b79f4dc573156bb160b8459b9d4a08207658eba4bb84c529dd4e02c7dee2c4ebcb37f1630d733a28dd91129182c79"
            ],
            "balance": "0x0",
            "codeHash": "0xc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470",
            "nonce": "0x0",
            "storageHash": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
            "storageProof": [
                {
                    "key": "0x0000000000000000000000000000000000000000000000000000000000000000",
                    "value": "0x0",
                    "proof": [],
                    "mutated": true
                },
                {
                    "key": "0x0000000000000000000000000000000000000000000000000000000000000003",
                    "value": "0x0",
                    "proof": [],
                    "mutated": false
                },
                {
                    "key": "0x0000000000000000000000000000000000000000000000000000000000000004",
                    "value": "0x0",
                    "proof": [],
                    "mutated": false
                },
                {
                    "key": "0x0000000000000000000000000000000000000000000000000000000000000001",
                    "value": "0x0",
                    "proof": [],
                    "mutated": true
                }
            ]
        },
        {
            "address": "0x4200000000000000000000000000000000000006",
            "accountProof": [
                "0xf843a0e881ac781cfca939c599937dce13fc740c981a83c94c2a6f0019a5ff1eacc97ba03075a5cfec7f6f8cc16a11b2050ffcd2580d88287b80c84a3bafc31efe1b2f6480",
                "0xf843a0351a139c15a6063a4c2d83739a4886e8294d84a1cd961153512465cd71cacceda0bcb845b54283b5b640b6b4b9c56137867dd43cf4aac491bd29bf7faf70271ad980",
                "0xf843a0c64184437278ab7917da8f734c25020b99cf87e1bc0eea2c4388f36fbb96c664a03f5aa0221bcaf63666dbeeb15ceb3f7212e09378245438a467ddec172500fd2380",
                "0xf843a0a4206ccb8a5b649d7b7740d35d699032f6f69359b086ebff43599b7bddb56e8aa0c6801d723545a08196db42f06ed7564e121f630595482d7f5deebd296f6b7abb80",
                "0xf843a0b9725113d3d912a79d1d4a94a54b64bf073cc54c52e92f732e2e3ae47b8c36dca03321c43b551c694d8ee608535ad07dc7fa8512f46f00fb8577a9141c0199798c80",
                "0xf843a0f35ee5ac004d6939689983c9128dc711f56a36279d2c879762b2e773099ad525a028e14e65cbc7ad582ddacc154cc0d04c51b20af71fe61dbb036f340d407cb55280",
                "0xf843a030d001d6cc9ca46ad62705f6db8b5d79629e3e52f2396ae57224cfb4a2988bb0a0e728d18846e394a5e5ff57c6480f67f549cf2cb9df3c083b48b8dec644e7404d80",
                "0xf843a057e8e3bdba1912bd574db05e9f0789e301ba9fafe5e937a1f86ef1736763eca8a0423c86bf5f6918f3820bb01e0dcd0cd3c119c1e1b92b02ed08139b974b6969fe80",
                "0xf843a023071354bba61ab96833c52286925bb5a8e603582743db51aebc55f86a575f19a034f2cfe18f5912f6bb431f2ffae321221f748e1937f1b1d2a206c8a8b73cee1080",
                "0xf843a0fff9290790d9f03d2081b4148379fe294ab04a73f8778d496938cfa09c9c74a2a07d179f55403c1066715312a618b7520ae4cff54af4f3bb0b1f103cbe5789e84080",
                "0xf843a03515e7be872ac031a314da4b13472ac17287a3c36b799495081f6f43fc6f7eaca0ba4ad8f350d65473e95c054a464f5a6e401554b30df00cfa12466efb133ac57980",
                "0xf869a0f93daad7526521f49e4a267f90a72e3b1871806fd705cd693c19b490d5251000b846f8448080a056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421a03d51cb4c14a5032f628f5e6cd20aebea0c2cb173617c3b2ef013692eb8ee47a3"
            ],
            "balance": "0x0",
            "codeHash": "0x3d51cb4c14a5032f628f5e6cd20aebea0c2cb173617c3b2ef013692eb8ee47a3",
            "nonce": "0x0",
            "storageHash": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
            "storageProof": [
                {
                    "key": "0xd4435d69fb8dfa20243856f7d55a4f6f87499860ebef0442d482d29b9b840a5b",
                    "value": "0x0",
                    "proof": [],
                    "mutated": false
                },
                {
                    "key": "0x8632b3478ce27e6c2251f16f71bf134373ff9d23cff5b8d5f95475fa6e52fe22",
                    "value": "0x0",
                    "proof": [],
                    "mutated": false
                }
            ]
        },
        {
            "address": "0xbae1ecd5647aca56326bfef7accc6720750f5a22",
            "accountProof": [
                "0xf843a0e881ac781cfca939c599937dce13fc740c981a83c94c2a6f0019a5ff1eacc97ba03075a5cfec7f6f8cc16a11b2050ffcd2580d88287b80c84a3bafc31efe1b2f6480",
                "0xf843a0351a139c15a6063a4c2d83739a4886e8294d84a1cd961153512465cd71cacceda0bcb845b54283b5b640b6b4b9c56137867dd43cf4aac491bd29bf7faf70271ad980",
                "0xf843a0c64184437278ab7917da8f734c25020b99cf87e1bc0eea2c4388f36fbb96c664a03f5aa0221bcaf63666dbeeb15ceb3f7212e09378245438a467ddec172500fd2380",
                "0xf843a0eef15e80e21d673fa86cf835d3086ce66518150e00ce8dbd4b00bb73a0e24132a0b71d6e6c4b4fbfdd67b640039d33b8d8025ffe25ca222b0fa26de8d08a61ba3580",
                "0xf843a0a72fbd949f87114db7370357fdf85a1ccb132d2a78bec4a421db8ad145c7ba0da02ede11c472c48937628287cb5bfd01675713cf0dc57c77acbb3fadede5c3607380",
                "0xf843a0ad6ccd10f054ca478bec2bad935646685492c019b2635e81efae04b34b4da1dca07b99f28c8f10b1e5c2ab9a767fae70b6d69e4ffbd7c7a430c52c9598dceef9dc80",
                "0xf843a05987be8f2d07c84eda8d97ff62be3da0423f5d3121575c34e6a14b8ced7d8f1aa01dfef3c94c73255bc56f102c86f2555770ab6a0aeb6e77c5af26b4fc153fd05f80",
                "0xf843a0e02b1276e6403d96d67980703fd52e377f16c94c3a1904e71a5404a785e9fbafa0d0cc63a77623c09bdd55ff3e496850ea79fd1f42a42e6f540724d6b18aaa676780",
                "0xe230a0609053ef433e1d35b2742ae5e3600652ec224d0cc57d7576214c371d164e501e",
                "0xf843a085369b0f0be366a6be3c7bb5c4bd6fec84d354db72de7f5ff892f7f037fb2555a05e43274e5e9a34e277e6dcffd0cbd2bf83b15482772295c38a4c0f3cf8f7b9c980",
                "0xf843a0852193fef48f6c8b61469868550143484cbed1c7efa40c650bfa924f21a5ae0da00551f886731fdb818bd9f6b06f6d4a3049f3dbf102c046bb1010c5cb37f087db80",
                "0xf869a0f7d7e771c8cf799a15567dae1a7625cb6deff893f6d3b73a09ba8714f4f9ce80b846f8448080a0d266be41e505c4aa98b1bd82cd7676571193ed7e571658e8a40da1897777efa4a0a799aac89d77d5783ca00ab6591670a05646eb19d18e0dc32f9d42d4d08ad19b"
            ],
            "balance": "0x0",
            "codeHash": "0xa799aac89d77d5783ca00ab6591670a05646eb19d18e0dc32f9d42d4d08ad19b",
            "nonce": "0x0",
            "storageHash": "0xd266be41e505c4aa98b1bd82cd7676571193ed7e571658e8a40da1897777efa4",
            "storageProof": [
                {
                    "key": "0x0000000000000000000000000000000000000000000000000000000000000000",
                    "value": "0x2",
                    "proof": [
                        "0xe3a1c290decd9548b62a8d60345a988386fc84ba6bc95484008f6362f93160ef3e563002"
                    ],
                    "mutated": false
                }
            ]
        }
    ]
}

[karlfloersch]

This is 🔥🔥🔥🔥🔥

Let me know if you have any particular questions or want to bounce ideas off of me during the short period of time where we are both awake 😄

What I'll do in the meantime I'll start sketching out an exact API that we can target for both the proof generator API that you're building & the proof submitter that @kfichter is owning.

I'll add it to the hackmd that we linked & actually both @kfichter & @gakonst y'all please feel free to fill in the API cuz i'm sure I'll be missing things

[gakonst]

@tynes @karlfloersch The only remaining thing is switching the in-memory map with something like leveldb. Due to the value of the k-v pair here being an array, it makes me think that leveldb might not be ideal here? Not sure.

[smartcontracts]

@gakonst we'll need to add "exclusion proofs" for trie elements that did not exist before the transaction. This can occur in the following cases:

• Contract creations
• Calls to empty accounts
• Changes to storage slots that were originally unset

"Exclusion proof" here refers to a proof that there is no element at a given key. Concretely, this is a partial inclusion proof that hits a dead end at some point down the trie. Does this PR already support that? Or does it need to be added? I don't believe we need to identify the difference between inclusion/exclusion proofs (can be handled in the contracts or determined client-side).

[gakonst]

@kfichter I think we already have support for that, since we use GetProof.

Contract Creation & Empty Accounts

The codehash for undeployed contracts or uninitialized accounts is always 0xc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470. When your client software finds such a codehash, it should take the address and the accountProof and call proveEmptyContractState, to prove that the account was empty during the pre-state root.

• If you need to further differentiate on if the empty account was an EOA or a contract, the storageHash of a non-contract account is 0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421 (while for contract it'll differ depending on bytecode), so you can use that as a flag client side.
• Separately, I was confused, why your EMPTY_CODE_HASH constant was 0x00004B1DC0DE000000004B1DC0DE000000004B1DC0DE000000004B1DC0DE0000 instead of 0xc5d..470?

Changes to contract slots that were originally unset

You want to prove that a deployed contract's slot during a pre-state root was unset. I believe that "unset" and "equals zero" is the same statement here, right? In that case, we're still covered by the returned proof.

Let's inspect some random storage slots for a Uniswap pair at block 0xa31457 (when the contract was deployed, if you inspect at 0xa31456, you'll see that everything is empty - as expected):

curl -s -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_getProof","params":["0x9346c20186d1794101b8517177a1b15c49c9ff9b", ["0x0", "0x99", "0x1123312"], "0xa31457"],"id":83}' <ARCHIVE NODE URL>

Response:

[
  {
    "key": "0x0",
    "proof": [
      "0xf8f1a02e2bc0c0ff22609eac8f10e1c8736f3e780dcb85055451e7ac674e2667ce4b5780a047d02852ca7a45b472ac744427294869d30a94545f5a4fe252b2852c76af7a5380808080808080a06bf06cedf7b90669bac0f199b18bceca612452bb315f1386645bfbd52205a47680a0739180a3fcc81e93aa247b3ad01095b73328073134789ef8cbc373196215e597a0303876dcca400618ab14b9d5ff6416ebfde7cd05bd17fd78cab31d79c2759187a02c1eacaf2e464f0d3f388e00a1ca096bacd3653f903a9f14bf06752d30190dd3a0b95483fd5777a01b9a964c61b87a5ebd2aa0508f2f2c4f9a77402028a144439d80",
      "0xeca0390decd9548b62a8d60345a988386fc84ba6bc95484008f6362f93160ef3e5638a891fec97671cf0463377"
    ],
    "value": "0x1fec97671cf0463377"
  },
  {
    "key": "0x99",
    "proof": [
      "0xf8f1a02e2bc0c0ff22609eac8f10e1c8736f3e780dcb85055451e7ac674e2667ce4b5780a047d02852ca7a45b472ac744427294869d30a94545f5a4fe252b2852c76af7a5380808080808080a06bf06cedf7b90669bac0f199b18bceca612452bb315f1386645bfbd52205a47680a0739180a3fcc81e93aa247b3ad01095b73328073134789ef8cbc373196215e597a0303876dcca400618ab14b9d5ff6416ebfde7cd05bd17fd78cab31d79c2759187a02c1eacaf2e464f0d3f388e00a1ca096bacd3653f903a9f14bf06752d30190dd3a0b95483fd5777a01b9a964c61b87a5ebd2aa0508f2f2c4f9a77402028a144439d80"
    ],
    "value": "0x0"
  },
  {
    "key": "0x1123312",
    "proof": [
      "0xf8f1a02e2bc0c0ff22609eac8f10e1c8736f3e780dcb85055451e7ac674e2667ce4b5780a047d02852ca7a45b472ac744427294869d30a94545f5a4fe252b2852c76af7a5380808080808080a06bf06cedf7b90669bac0f199b18bceca612452bb315f1386645bfbd52205a47680a0739180a3fcc81e93aa247b3ad01095b73328073134789ef8cbc373196215e597a0303876dcca400618ab14b9d5ff6416ebfde7cd05bd17fd78cab31d79c2759187a02c1eacaf2e464f0d3f388e00a1ca096bacd3653f903a9f14bf06752d30190dd3a0b95483fd5777a01b9a964c61b87a5ebd2aa0508f2f2c4f9a77402028a144439d80",
      "0xf7a0366cc928b5edb82af9bd49922954155ab7b0942694bea4ce44661d9a8736c6889594c02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"
    ],
    "value": "0x0"
  }
]

You can see that even though these random slots are uninitialized, they have a non-empty proof which you can use to prove that the value was 0 in the pre-state, and non-zero in the post-state.

[tynes]

We did add a new RPC namespace rollup, I believe the PublicBlockChainAPI will expose in the eth namespace so it'll be eth_getStateDiffProof. Would users expect this to be in the eth namespace or rollup namespace? See:

go-ethereum/internal/ethapi/api.go

Line 1684 in 08903b8

func NewPublicRollupAPI(b Backend) *PublicRollupAPI {

[gakonst]

No strong preference, I think either works. I decided to add it to eth's namespace to be next to GetProof

[smartcontracts]

@gakonst I noticed something very weird while testing out this branch. It seems that state roots are somehow being changed after a block is mined. I have no clue why this might be. For context, our "state batch submitter" is calling eth_getBlockByNumber in order to submit batches of state roots. When it does this, the service will get some result for the value of the state root. When I later call eth_getBlockByNumber for the same state root within the "fraud prover," we get a completely different root. Any idea why this might be happening?

[gakonst]

@kfichter is your issue reproducible on master? If yes, it's prob unrelated to this PR. If not, could you share the repro code so I can investigate? My code does not touch that functionality, so I agree that it's weird.

[gakonst]

@kfichter confirmed out of band that the described issue is unrelated to this PR.

PR is good for another review round

[smartcontracts]

@gakonst I noticed that a few contracts aren't being included in the proof, in particular:

• The EOA account contract.
• The ECDSA account precompile (0x42....0003)
• The sequencer entrypoint precompile (0x42....0005)
• And, for some reason, one particular storage slot for the "gas metadata address" (0x06a506a5....) -- slot 0x00...0002.

Last one might just point to a discrepancy in L1 and L2 execution. Precompiles probably don't need to be included in the proofs, so maybe not necessary to cover 2 and 3 (maybe @ben-chain can confirm?). First one seems crucial though, not sure why it isn't included right now.

Thank you for all the work here!!

[gakonst]

I wonder if all of these do not appear in the proof due to testing directly via L2. Perhaps doing a L1 -> L2 message test would help here (Not sure how? Is there some E2E example?). @ben-chain it'd be great if we can get your input

[ben-chain]

We def need proof for bullet 1., and I am pretty confident we also need account proofs for #2 and #3 "ovm precompiles" (I've been trying to get in the habit of calling these "ovm predeploys" to avoid name collision with EVM precompiles). These precompiles are not actually treated differently than other OVM contracts, they are just put into the genesis state at known addresses. We could introduce custom logic to avoid, but I think that configurability is not worth it ATM. Therefore I think the first three are definitely critical.

Agreed that the last one is needed but only in some cases and not getting it could be a diff in L1/L2 execution. I'll just enumerate the scenarios which 0x06a506a5..., 0x00...0002 would be needed:

1. Any time there is a new epoch start in that tx (in this case, all 5 slots are needed)
2. Any transaction where the queue origin is L1->L2 (NOT a sequencer tx)

If this is a sequencer tx, and 1. is not the case, then it must be we are incorrectly setting the queue origin in geth.

[smartcontracts]

Should be return s.b.GetDiff(new(big.Int).Add(header.Number, big.NewInt(1)))

[smartcontracts]

Because eth_getProof returns a proof for the state after the given block is executed. So if the state after block N is our "pre-state" then we want to get the state diff for block N+1.

[smartcontracts]

Other option is to instead do eth_getProof for block N-1 and keep this line unchanged

[gakonst]

If block N is our pre-state and block N+1 is our post state, then we should call it with N and N+1, no? Seems correct to me

[tynes]

Is this dead code?

[tynes]

I think this is ready to merge, @kfichter has been consuming it so will merge if he gives the thumbs up