Implement eth/64 protocol by cburgdorf · Pull Request #1543 · ethereum/trinity

cburgdorf · 2020-02-10T14:25:58Z

What was wrong?

Trinity is currently on eth/63 whereas Geth has implemented eth/64 and eth/65 already. We are pushing for the inclusion of request ids as eth/66.

Before we add an implementation for the proposed eth/66 protocol, we should first implement eth/64 and eth/65.

How was it fixed?

This implements the eth/64 protocol according to EIP 2124. The implementation follows the general principle of how LESV1, LESV2 are implemented side by side.

This also adds a tiny cleanup to combine three different places with the same if/else construct to choose the concrete API version into a single helper function.

Real life testing also demonstrates we are having eth/64 peers alongside eth/63 peers in the pool.

Adding ETHPeer (eth, 64) <Session <Node(0x089581@45.40.132.135)> 1da28744-11a1-4ce4-82e3-c84449cd41df> to pool
Adding ETHPeer (eth, 63) <Session <Node(0x5629f4@54.39.133.125)> 656b4238-5afd-4df8-8f29-515022183c0b> to pool

To-Do

Add entry to the release notes

Cute Animal Picture

pipermerriam

If you can show that ETH/64 is supported our-of-the-box by the majority of network nodes (I don't know whether it actually is...) then we can drop ETH/63

cburgdorf · 2020-02-11T08:41:30Z

If you can show that ETH/64 is supported our-of-the-box by the majority of network nodes (I don't know whether it actually is...) then we can drop ETH/63

Yeah, I thought about that too but ended up abandoning the thought because:

Geth still supports eth/63
It's a good code exercise to support two different handshakes especially when we already remove LES

cburgdorf · 2020-02-14T14:17:22Z

Alright, this should be ready for review.

cburgdorf · 2020-02-19T09:46:42Z

I have reassigned this to you @gsalgado because I think @pipermerriam is very busy right now.

gsalgado

LGTM, just a few comments/suggestions

gsalgado · 2020-02-20T03:58:59Z

tests/core/p2p-proto/test_eth_api.py

+    return chain
+
+
+@pytest.fixture(params=(ETHV63PeerPairFactory, ETHPeerPairFactory))


So, this causes all fixtures that use this one to be parameterized as well, I guess?

Correct! I tried to keep the tests structure similar to the LESV1 vs LESV2 tests that use the same pattern?

gsalgado · 2020-02-20T04:00:23Z

tests/core/p2p-proto/test_eth_api.py

    ChainContextFactory,
    ETHPeerPairFactory,
 )
+from trinity.tools.factories.eth.proto import ETHV63PeerPairFactory


Why is this factory not imported from the same place as ETHPeerPairFactory?

This is me testing out PyCharm Professional and relying on its auto import feature 😅 I'll fix that.

gsalgado · 2020-02-20T04:05:27Z

trinity/protocol/eth/api.py

+        self.protocol.send(StatusV63(payload))
+
+
+class ETHAPI(BaseETHAPI):


Why did you chose not to include the version in the class name here?

I have only put in the version numbers in classes that aren't on the latest version.

E.g.

ETHV63API ETHProtocolV63 ETHV63PeerFactory

vs

ETHAPI ETHProtocol ETHPeerFactory

gsalgado · 2020-02-20T04:12:50Z

trinity/protocol/eth/handshaker.py

+                raise WrongForkIDFailure(
+                    f"{multiplexer.remote} network "
+                    f"({receipt.handshake_params.fork_id}) is incompatible to ours "
+                    f"({self.handshake_params.fork_id}), disconnecting"


Maybe include the actual exception msg here as well, as it would give extra details about where exactly the incompatibility lies.

gsalgado · 2020-02-20T04:13:04Z

trinity/protocol/eth/handshaker.py

+                )
+            except BaseForkIDValidationError:
+                raise WrongForkIDFailure(
+                    f"{multiplexer.remote} network "


s/network/forkid?

https://eips.ethereum.org/EIPS/eip-2124

cburgdorf · 2020-02-20T09:26:30Z

Just ran a last live test and had this error blasting out.

ERROR  2020-02-20 10:20:24,392           ETHPeerPool  unexpected error during peer connection
Traceback (most recent call last):
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/commands.py", line 131, in decode
    payload = cls.serialization_codec.decode(payload_data)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/commands.py", line 64, in decode
    rlp.decode(data, strict=self.decode_strict, sedes=self.sedes, recursive_cache=True)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/venv/lib/python3.7/site-packages/rlp/codec.py", line 237, in decode
    obj = sedes.deserialize(item, **kwargs)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/venv/lib/python3.7/site-packages/eth_utils/functional.py", line 45, in inner
    return callback(fn(*args, **kwargs))
  File "/home/cburgdorf/Documents/hacking/ef/trinity/venv/lib/python3.7/site-packages/rlp/sedes/lists.py", line 89, in deserialize
    serial)
rlp.exceptions.ListDeserializationError: Deserializing list length (5) does not match sedes (6)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/cburgdorf/Documents/hacking/ef/trinity/trinity/protocol/common/peer.py", line 206, in maybe_connect_more_peers
    for backend in self.peer_backends
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/peer_pool.py", line 209, in _add_peers_from_backend
    await self.connect_to_nodes(iter(candidates))
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/peer_pool.py", line 401, in connect_to_nodes
    loop=self.get_event_loop(),
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/peer_pool.py", line 437, in connect_to_node
    peer = await self.connect(node)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/peer_pool.py", line 347, in connect
    timeout=HANDSHAKE_TIMEOUT,
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/cancellable.py", line 20, in wait
    return await self.wait_first(awaitable, token=token, timeout=timeout)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/cancellable.py", line 43, in wait_first
    return await token_chain.cancellable_wait(*awaitables, timeout=timeout)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/venv/lib/python3.7/site-packages/cancel_token/token.py", line 163, in cancellable_wait
    await future
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/peer.py", line 492, in handshake
    token=self.cancel_token
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/handshake.py", line 323, in dial_out
    token=token,
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/handshake.py", line 288, in negotiate_protocol_handshakes
    in zip(selected_handshakers, selected_protocols)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/venv/lib/python3.7/site-packages/async_generator/_util.py", line 53, in __aexit__
    await self._agen.athrow(type, value, traceback)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/multiplexer.py", line 381, in multiplex
    fut.result()
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/multiplexer.py", line 404, in _do_multiplexing
    await self._handle_commands(msg_stream, stop)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/multiplexer.py", line 421, in _handle_commands
    async for protocol, cmd in msg_stream:
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/cancellable.py", line 67, in wait_iter
    timeout=timeout,
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/cancellable.py", line 20, in wait
    return await self.wait_first(awaitable, token=token, timeout=timeout)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/cancellable.py", line 43, in wait_first
    return await token_chain.cancellable_wait(*awaitables, timeout=timeout)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/venv/lib/python3.7/site-packages/cancel_token/token.py", line 178, in cancellable_wait
    return done.pop().result()
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/multiplexer.py", line 87, in stream_transport_messages
    cmd = command_type.decode(msg, msg_proto.snappy_support)
  File "/home/cburgdorf/Documents/hacking/ef/trinity/p2p/commands.py", line 136, in decode
    ) from err
rlp.exceptions.DeserializationError: DeserializationError for <class 'trinity.protocol.eth.commands.Status'>

I had live tested this before with a mix of eth/64 and eth/63 peers and had no errors. Need to check where this is coming from. It looks like someone is sending us an eth/63Status message where we expect an eth/64 one.

So I need to hold of merging this until I figured out what's wrong here.

cburgdorf · 2020-02-21T17:04:38Z

@gsalgado Mind taking a look at the most recent commit that I added? This deals with the DeserializationError. From all I can tell, this has nothing to do with this PR per se. I think what happens is that we hit this error if we try to do a handshake with a client that advertises eth/64 but then actually responds with the wrong status message. This is now treated as a MalformedMessage.

gsalgado · 2020-02-23T22:29:17Z

p2p/multiplexer.py

+        try:
+            cmd = command_type.decode(msg, msg_proto.snappy_support)
+        except rlp.exceptions.DeserializationError:
+            raise MalformedMessage(f"Failed to decode {msg} for {command_type}")


Maybe use raise MalformedMessage(...) from err, to include the original error?

pipermerriam reviewed Feb 10, 2020

View reviewed changes

cburgdorf force-pushed the christoph/feat/eth64 branch 13 times, most recently from 65bccfc to dcf264b Compare February 14, 2020 13:52

cburgdorf requested review from gsalgado and pipermerriam February 14, 2020 14:17

gsalgado reviewed Feb 20, 2020

View reviewed changes

cburgdorf added 4 commits February 20, 2020 10:13

Make HanshakerAPI generic over TProtocol

f76e2d0

Rename ETHProtocol to ETHProtocolV63

3755626

Implement eth/64 protocol according to EIP-2124

aad1924

https://eips.ethereum.org/EIPS/eip-2124

Refactor code to be more DRY

0a5a9f3

cburgdorf force-pushed the christoph/feat/eth64 branch from dcf264b to 0a5a9f3 Compare February 20, 2020 09:16

gsalgado reviewed Feb 23, 2020

View reviewed changes

Turn DeserializationError into MalformedMessage

0f1ecd0

cburgdorf force-pushed the christoph/feat/eth64 branch from 732c53f to 0f1ecd0 Compare February 24, 2020 09:55

cburgdorf merged commit f2d1bb4 into ethereum:master Feb 24, 2020

		return chain


		@pytest.fixture(params=(ETHV63PeerPairFactory, ETHPeerPairFactory))

		self.protocol.send(StatusV63(payload))


		class ETHAPI(BaseETHAPI):

Conversation

cburgdorf commented Feb 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What was wrong?

How was it fixed?

To-Do

Cute Animal Picture

Uh oh!

pipermerriam left a comment

Choose a reason for hiding this comment

Uh oh!

cburgdorf commented Feb 11, 2020

Uh oh!

cburgdorf commented Feb 14, 2020

Uh oh!

cburgdorf commented Feb 19, 2020

Uh oh!

gsalgado left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cburgdorf Feb 20, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cburgdorf commented Feb 20, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cburgdorf commented Feb 21, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cburgdorf commented Feb 10, 2020 •

edited

Loading

cburgdorf Feb 20, 2020 •

edited

Loading

cburgdorf commented Feb 20, 2020 •

edited

Loading