p2p: peer metainfo support

[algorandskiy]

Summary

P2P network was missing peer meta info exchange including peer features like vote compression.

General idea: current p2p /algorand-ws/1.0.0 handler has a stub for meta info exchange (handshake) similar to HTTP: connection initiator writes a single byte, server reads a byte. Unfortunately there is no elegant solution for keeping a single /algorand-ws/1.0.0 but also support both old single byte handshake and multibyte meta info exchange when both old and new node have to communicate each other. The solution is to add a new p2p /algorand-ws/2.2.0 proto with multibyte handshake consisting of 2 bytes of data length and msgp-encoded http.Header-like map as a data portion. p2p nodes can list remote protocols and determine either /algorand-ws/2.2.0 or /algorand-ws/1.0.0 to use.

In this PR:

1. Generalize setHeader to be able to use in both wsnet and p2pnet
2. Add /algorand-ws/2.2.0 p2p proto in addition to /algorand-ws/1.0.0
3. Add meta info exchange into /algorand-ws/2.2.0 handler
4. Use ordered list of protocols in NewStream and let libp2p make a selection. Switch from network's Connected notification to EvtPeerIdentificationCompleted in order to be able to communicate with both old /algorand-ws/1.0.0 and new /algorand-ws/2.2.0 nodes
5. Removed the old telemetryID exchange from p2p since it is racy

Test Plan

Added unit tests:

1. meta info read write functions
2. p2p nodes meta info exchange
3. nodes of different net proto versions

Manual test:
Run a new p2p node vs betanet and testnet to ensure it can fast catchup, catchup and follow the chain.

[codecov]

Codecov Report

Attention: Patch coverage is 68.94977% with 68 lines in your changes missing coverage. Please review.

Project coverage is 51.05%. Comparing base (005bf7e) to head (6655439).
Report is 7 commits behind head on master.

Files with missing lines	Patch %	Lines
network/p2p/streams.go	0.00%	32 Missing ⚠️
network/p2pNetwork.go	70.27%	15 Missing and 7 partials ⚠️
network/wsNetwork.go	86.79%	5 Missing and 2 partials ⚠️
network/p2p/p2p.go	0.00%	4 Missing ⚠️
network/p2pMetainfo.go	94.54%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6312      +/-   ##
==========================================
- Coverage   51.85%   51.05%   -0.81%     
==========================================
  Files         652      653       +1     
  Lines       87433    87549     +116     
==========================================
- Hits        45340    44694     -646     
- Misses      39226    39932     +706     
- Partials     2867     2923      +56     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull Request Overview

This PR adds peer meta info exchange support for p2p networks and generalizes header setting to work across both wsnet and p2pnet. Key changes include:

• Renaming of identifier fields (e.g. GenesisID/RandomID to genesisID/randomID) and updating their usages.
• Refactoring of header-setting into a generic function using a peerMetadataProvider interface.
• Addition of support for the new /algorand-ws/2.2.0 protocol in p2p stream handlers and adjusted test timeouts to aid stability in race builds.

Reviewed Changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
node/node_test.go	Updated log messages to include peer IDs in phonebook responses.
network/wsNetwork.go	Transitioned from public to lowercase identifier fields and refactored header setting.
network/p2pNetwork.go	Introduced support for dual protocol versions and temporary logic with a TODO comment.
network/*_test.go	Extended test timeouts for p2p tests to mitigate race conditions.
network/streams.go	Refactored stream dispatch to use protocol-specific handler maps.
network/msgp_gen*.go	Regenerated msgp code for peer meta headers/values.

Comments suppressed due to low confidence (3)

network/p2pNetwork_test.go:109

• Increasing the timeout from 2 to 5 seconds can help with race conditions on CI builds, but please confirm that the longer wait is intentional and does not hide potential performance issues.

}, 5*time.Second, 50*time.Millisecond)

network/p2pNetwork.go:292

• The temporary protocol version selection logic gated by the disableV22Protocol flag is clearly marked with a TODO; ensure that this block is removed once consensus v41 is deployed.

TODO: remove after consensus v41 takes effect.

node/node_test.go:1025

• [nitpick] Please verify that using ni[0].p2pID for node 0 is the intended behavior and is consistent with the later cases that log ni[1].p2pID and ni[2].p2pID.

t.Logf("Node%d %s phonebook: %s", i, ni[0].p2pID, ni[1].p2pMultiAddr())

[algorandskiy]

Posted some notes for reviewers

[gmalouf]

Left some initial comments, my biggest concerns are the compatibility issues.

[algorandskiy]

my biggest concerns are the compatibility issues.

that's the point of an extra p2p proto to have it fully backward compatible (telemetryID stuff is removed b/c it non-working I believe in most cases).

[jannotti]

I am only getting started understanding how this stuff works, but it looks to me like in v1, there is a single byte exchange, but neither side cares what the bytes actually are. It looks like they send ascii "1".

Why couldn't we say "if you send a certain byte, the headers come next"?

[algorandskiy]

Why couldn't we say "if you send a certain byte, the headers come next"?

because of the following: new client and old server combo: old server can only read one byte and write one byte, and new client initiates the exchange by writing something not knowing what other side expects - one byte or multiple bytes.
Ok, new client can write a byte and read with timeout, if nothing came out then guess it is a new server and push meta info. But it brings 1) connection delay 2) more complex logic rather then two well separated p2p protocols.

[jannotti]

Right, of course. Thanks!

…

On Tue, May 13, 2025 at 12:29 PM Pavel Zbitskiy ***@***.***> wrote: *algorandskiy* left a comment (algorand/go-algorand#6312) <#6312 (comment)> Why couldn't we say "if you send a certain byte, the headers come next"? because of the following: new client and old server combo: old server can only read one byte and write one byte, and new client initiates the exchange by writing something not knowing what other side expects - one byte or multiple bytes. Ok, new client can write a byte and read with timeout, if nothing came out then guess it is a new server and push meta info. But it brings 1) connection delay 2) more complex logic rather then two well separated p2p protocols. — Reply to this email directly, view it on GitHub <#6312 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADL7T7XYKDOJCHUPZQVYZ326IMW5AVCNFSM6AAAAAB4LFDI4WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNZXGIYDQOBYGU> . You are receiving this because your review was requested.Message ID: ***@***.***>

[jannotti]

I like the simplification