Skip to content
This repository has been archived by the owner on Oct 19, 2024. It is now read-only.

Transaction::access_list set to None is incorrectly RLP encoded #1134

Closed
michaelsproul opened this issue Apr 11, 2022 · 2 comments · Fixed by #1137
Closed

Transaction::access_list set to None is incorrectly RLP encoded #1134

michaelsproul opened this issue Apr 11, 2022 · 2 comments · Fixed by #1137
Labels
bug Something isn't working

Comments

@michaelsproul
Copy link
Contributor

michaelsproul commented Apr 11, 2022

Version

ethers-core @ 1d14f9d

Platform

Linux xxx 5.13.0-39-generic #44-Ubuntu SMP Thu Mar 24 15:35:05 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Description

I believe None access lists in Transactions are being encoded incorrectly, by encoding them as empty strings rather than empty lists here:

rlp_opt(&mut rlp, &self.access_list);

I think the fix might look something like:

rlp.append(&self.access_list.unwrap_or(&[]));

The relevant part of the RLP spec reads:

If the total payload of a list (i.e. the combined length of all its items being RLP encoded) is 0-55 bytes long, the RLP encoding consists of a single byte with value 0xc0 plus the length of the list followed by the concatenation of the RLP encodings of the items. The range of the first byte is thus [0xc0, 0xf7].

https://eth.wiki/fundamentals/rlp

i.e. an empty access list should be 0xc0 not 0x80.

Example

I found this while attempting to re-serialize transactions in Lighthouse on the Kiln merge testnet.

One prototypical transaction is this one, serialized by ethers-rs and Nethermind:

{
        "hash": "0x70e6d41854fe4f50731184371c32d4a2c8bb83c47579ef200dc64c702136fac0",
        "nonce": "0x91",
        "blockHash": "0x892e8efabaae838a689c711789c6b18883ef6fc4c31ec0651f6a13012680dc6c",
        "blockNumber": "0x33828",
        "transactionIndex": "0x0",
        "from": "0x818306150064f0f091c46d4cccaa288b0976ad37",
        "to": "0xd1003b592859d5c0241d7c1f744d64d9d043cef4",
        "value": "0x0",
        "gasPrice": "0x77359407",
        "maxPriorityFeePerGas": "0x77359400",
        "maxFeePerGas": "0x16d6f91027",
        "gas": "0x3b9f2",
        "data": "0x32db54700000000000000000000000000000000000000000000000000000000000033810000000000000000000000000000000000000000000000007370d998df33659000000000000000000000000000000000000000000000000029b4d68f4ef41590000000000000000000000000000000000000000000000000735e2724866b50000",
        "input": "0x32db54700000000000000000000000000000000000000000000000000000000000033810000000000000000000000000000000000000000000000007370d998df33659000000000000000000000000000000000000000000000000029b4d68f4ef41590000000000000000000000000000000000000000000000000735e2724866b50000",
        "chainId": "0x1469ca",
        "type": "0x2",
        "v": "0x0",
        "s": "0x336ec99f0438391d55d9d4e2a52e06e2c9080db987c073966b481415eb0e6846",
        "r": "0x53df3e98bf698f1ce192c7d7452080d793ec4a4b6bbce7b4edf7718e77d7696d",
        "yParity": "0x0"
}

Ethers:

0x02f8f5831469ca819184773594008516d6f910278303b9f294d1003b592859d5c0241d7c1f744d64d9d043cef480b88432db54700000000000000000000000000000000000000000000000000000000000033810000000000000000000000000000000000000000000000007370d998df33659000000000000000000000000000000000000000000000000029b4d68f4ef41590000000000000000000000000000000000000000000000000735e2724866b500008080a053df3e98bf698f1ce192c7d7452080d793ec4a4b6bbce7b4edf7718e77d7696da0336ec99f0438391d55d9d4e2a52e06e2c9080db987c073966b481415eb0e6846

Nethermind:

0x02f8f5831469ca819184773594008516d6f910278303b9f294d1003b592859d5c0241d7c1f744d64d9d043cef480b88432db54700000000000000000000000000000000000000000000000000000000000033810000000000000000000000000000000000000000000000007370d998df33659000000000000000000000000000000000000000000000000029b4d68f4ef41590000000000000000000000000000000000000000000000000735e2724866b50000c080a053df3e98bf698f1ce192c7d7452080d793ec4a4b6bbce7b4edf7718e77d7696da0336ec99f0438391d55d9d4e2a52e06e2c9080db987c073966b481415eb0e6846
@michaelsproul michaelsproul added the bug Something isn't working label Apr 11, 2022
@michaelsproul
Copy link
Contributor Author

I'll have a go at raising a PR with tests to fix this tomorrow.

@gakonst
Copy link
Owner

gakonst commented Apr 11, 2022

cc @wolflo

michaelsproul added a commit to michaelsproul/ethers-rs that referenced this issue Apr 12, 2022
The optional access list on `Transaction` was being incorrectly encoded
as an empty string (0x80) when omitted, when it should be encoded as an
empty list (0xc0).

Fixes gakonst#1134.
michaelsproul added a commit to michaelsproul/ethers-rs that referenced this issue Apr 12, 2022
The optional access list on `Transaction` was being incorrectly encoded
as an empty string (0x80) when omitted, when it should be encoded as an
empty list (0xc0).

Fixes gakonst#1134.
gakonst pushed a commit that referenced this issue Apr 13, 2022
The optional access list on `Transaction` was being incorrectly encoded
as an empty string (0x80) when omitted, when it should be encoded as an
empty list (0xc0).

Fixes #1134.
bors bot pushed a commit to sigp/lighthouse that referenced this issue May 12, 2022
## Proposed Changes

Reduce post-merge disk usage by not storing finalized execution payloads in Lighthouse's database.

:warning: **This is achieved in a backwards-incompatible way for networks that have already merged** :warning:. Kiln users and shadow fork enjoyers will be unable to downgrade after running the code from this PR. The upgrade migration may take several minutes to run, and can't be aborted after it begins.

The main changes are:

- New column in the database called `ExecPayload`, keyed by beacon block root.
- The `BeaconBlock` column now stores blinded blocks only.
- Lots of places that previously used full blocks now use blinded blocks, e.g. analytics APIs, block replay in the DB, etc.
- On finalization:
    - `prune_abanonded_forks` deletes non-canonical payloads whilst deleting non-canonical blocks.
    - `migrate_db` deletes finalized canonical payloads whilst deleting finalized states.
- Conversions between blinded and full blocks are implemented in a compositional way, duplicating some work from Sean's PR #3134.
- The execution layer has a new `get_payload_by_block_hash` method that reconstructs a payload using the EE's `eth_getBlockByHash` call.
   - I've tested manually that it works on Kiln, using Geth and Nethermind.
   - This isn't necessarily the most efficient method, and new engine APIs are being discussed to improve this: ethereum/execution-apis#146.
   - We're depending on the `ethers` master branch, due to lots of recent changes. We're also using a workaround for gakonst/ethers-rs#1134.
- Payload reconstruction is used in the HTTP API via `BeaconChain::get_block`, which is now `async`. Due to the `async` fn, the `blocking_json` wrapper has been removed.
- Payload reconstruction is used in network RPC to serve blocks-by-{root,range} responses. Here the `async` adjustment is messier, although I think I've managed to come up with a reasonable compromise: the handlers take the `SendOnDrop` by value so that they can drop it on _task completion_ (after the `fn` returns). Still, this is introducing disk reads onto core executor threads, which may have a negative performance impact (thoughts appreciated).

## Additional Info

- [x] For performance it would be great to remove the cloning of full blocks when converting them to blinded blocks to write to disk. I'm going to experiment with a `put_block` API that takes the block by value, breaks it into a blinded block and a payload, stores the blinded block, and then re-assembles the full block for the caller.
- [x] We should measure the latency of blocks-by-root and blocks-by-range responses.
- [x] We should add integration tests that stress the payload reconstruction (basic tests done, issue for more extensive tests: #3159)
- [x] We should (manually) test the schema v9 migration from several prior versions, particularly as blocks have changed on disk and some migrations rely on being able to load blocks.

Co-authored-by: Paul Hauner <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants