Feat/stackerdb messages

[jcnelson]

This PR implements the message codes and handshake protocol for nodes that support Stacker DBs. This is part of an effort to split #3534 into a series of smaller PRs.

[codecov]

Codecov Report

Merging #3551 (5c84ee1) into develop (09ec5d3) will decrease coverage by 0.01%.
Report is 1 commits behind head on develop.
The diff coverage is 0.00%.

@@             Coverage Diff             @@
##           develop    #3551      +/-   ##
===========================================
- Coverage     0.18%    0.18%   -0.01%     
===========================================
  Files          306      306              
  Lines       278640   280746    +2106     
===========================================
  Hits           512      512              
- Misses      278128   280234    +2106     

Files Changed	Coverage Δ
src/burnchains/mod.rs	0.00% <ø> (ø)
src/chainstate/burn/db/sortdb.rs	0.00% <0.00%> (ø)
src/net/chat.rs	0.00% <ø> (ø)
src/net/codec.rs	0.00% <0.00%> (ø)
src/net/connection.rs	0.00% <0.00%> (ø)
src/net/inv.rs	0.00% <0.00%> (ø)
src/net/mod.rs	0.00% <0.00%> (ø)
src/net/p2p.rs	0.00% <0.00%> (ø)
stacks-common/src/codec/mod.rs	0.00% <0.00%> (ø)
testnet/stacks-node/src/neon_node.rs	0.00% <0.00%> (ø)
... and 1 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[kantai]

Can you add some rustdocs that provide an overview of what this new protocol is for, how it should work, and how it fits into the rest of the networking code? Otherwise, this is pretty difficult to provide a thorough review.

[jcnelson]

Can you add some rustdocs that provide an overview of what this new protocol is for, how it should work, and how it fits into the rest of the networking code? Otherwise, this is pretty difficult to provide a thorough review.

Yes; I just added some in c2d3ebd

[kantai]

This looks good to me -- I just had some feedback about comments, and then about the ContractId wrapper struct (I think should be replaced with an extension trait).

[jcnelson]

@kantai Thanks for your review! All of your comments have been addressed. Please let me know what you think 🙏

[kantai]

LGTM

[kantai]

Looking back at this PR now that it's active again -- is there test coverage for the error branches here? (either this invalid relayers branch or the bandwidth exceeded branch)?

[jcnelson]

The process_relayers() function is tested in the test convo_process_relayers() already.

[jcnelson]

Would that be sufficient for merge?

[kantai]

That's fine for process_relayers(), but there's many error cases in this changeset:

        if self.connection.options.max_stackerdb_push_bandwidth > 0
            && self.stats.get_stackerdb_push_bandwidth()
                > (self.connection.options.max_stackerdb_push_bandwidth as f64)

            StacksMessageType::StackerDBHandshakeAccept(ref data, ref db_data) => {
                if solicited {
                    test_debug!("{:?}: Got unauthenticated StackerDBHandshakeAccept", &self);
                    self.handle_handshake_accept(burnchain_view, &msg.preamble, data, Some(db_data))
                        .and_then(|_| Ok(None))
                } else {
                    test_debug!(
                        "{:?}: Unsolicited unauthenticated StackerDBHandshakeAccept",
                        &self
                    );

                    // don't update stats or state, and don't pass back
                    consume = true;
                    Ok(None)
                }

 match self.validate_stackerdb_push(
                    local_peer,
                    chain_view,
                    &msg.preamble,
                    msg.relayers.clone(),
                )? {
                    Some(handle) => Ok(handle),
                    None => {
                        // will forward upstream
                        return Ok(Some(msg));
                    }
                }

And for this branch, are both conditions tested (i.e., node supports "stacker dbs", but not the same stacker db as the handshake initializer?)

        let stacks_message = if ConversationP2P::supports_stackerdb(local_peer.services)
            && ConversationP2P::supports_stackerdb(self.peer_services)
        {

[jcnelson]

Sure; added a bunch of missing test coverage

[obycode]

This looks good to me, aside from those minor changes I suggested in the documentation, and assuming some tests will get added in src/net/stackerdb/tests/mod.rs in future PRs.

[kantai]

Okay, with the latest tests, this looks good to me. Before merging, be sure that you want this in develop as opposed to next (as in, this networking logic should ship in the next minor version release: 2.4.0.1.0). You should also update the CHANGELOG.md (also please note if this is a minor or major version release).