feat(network): upgrade p2p layer

[onur-ozkan]

This PR brings new p2p layer that uses the latest stable libp2p version and deprecates the old p2p layer.

This PR is blocker for the following future works:

• Gradually reduce the patches on komodo p2p fork (start with activating https://github.com/KomodoPlatform/komodo-defi-framework/blob/092d562f476ece1bef3cf3e8e50694e23688dade/mm2src/mm2_p2p/src/behaviours/peer_store.rs or relay,keep-alive functionalities)
• Seperate network layer from mm2 core/swap mechanism so mm2 can be used with any network implementation
• Message compression(as a new behaviour/protocol so we don't face with breaking changes) in p2p layer
• Implement rate limiter behaviour/protocol
• feat(network): upgrade p2p layer #1878 (comment)
• feat(network): upgrade p2p layer #1878 (comment)

Breaking changes:

• mesh protocols are upgraded to latest versions (like [email protected])

• latest libp2p requires using >=23 android ndk for aarch64
  

  komodo-defi-framework/scripts/ci/android-ndk.sh

  Lines 5 to 6 in 092d562

  	NDK_URL=https://dl.google.com/android/repository/android-ndk-r23c-linux.zip
  	NDK_CHECKSUM=6ce94604b77d28113ecd588d425363624a5228d9662450c48d2e4053f8039242

• Old mesh configuration is no more valid for latest version
  

  komodo-defi-framework/mm2src/mm2_p2p/src/behaviours/atomicdex.rs

  Line 595 in 0c00797

  let (mesh_n_low, mesh_n, mesh_n_high) = if i_am_relay { (4, 8, 12) } else { (2, 4, 6) };

• Migrated to modern straem multiplexer yamux
  

  komodo-defi-framework/mm2src/mm2_p2p/src/behaviours/atomicdex.rs

  Lines 917 to 920 in 0c00797

  	transport
  	.upgrade(libp2p::core::upgrade::Version::V1)
  	.authenticate(noise_config)
  	.multiplex(libp2p::yamux::Config::default())

• For gossipsub, now we broadcast one topic at a time
  

  komodo-defi-framework/mm2src/mm2_main/src/lp_network.rs

  Line 276 in 0c00797

  pub fn broadcast_p2p_msg(ctx: &MmArc, topic: String, msg: Vec<u8>, from: Option<PeerId>) {

---

Check Cargo.lock with display rich diff button for detailed dependency report.

[onur-ozkan]

AFAIK you are planning a specific release for this work. Do you want me to bump mm2 to 1.1.0-beta? @shamardy

[shamardy]

Do you want me to bump mm2 to 1.1.0-beta?

Yes, please. thanks for remembering.

Following semantic versioning a release including this PR should be 2.0.0-beta. But I will use the second digit since we only use the last digit (PATCH version) for backward compatible functionalities too. After the 1.1.0-beta release we will start using the standard semantic versioning.

[laruh]

@onur-ozkan excellent work! could you please add doc comments for pub items (at least for new rs files in mm2_p2p crate, if you have time), so we can start code preparation for missing_docs lint

[onur-ozkan]

@onur-ozkan excellent work! could you please add doc comments for pub items (at least for new rs files in mm2_p2p crate, if you have time), so we can start code preparation for missing_docs lint

PR is already under sec-review and about to be merged. Would be much nicer if we ignore those crates with #![allow(missing_docs)], which can be fixed later.

[ca333]

secure code reviewed (based on cargo.lock / ensured pinned V >= cargo deps V / re-reviewed prior utilized deps from this list) - no malicious bits detected

general deps:

async-trait @ 0.1.52 (https://github.com/dtolnay/async-trait/tree/0.1.52 / 20bd296e0d646e2b14626b7078e045254bed26ee) outdated

derive_more @ 0.99.11 (https://github.com/JelteF/derive_more/tree/v0.99.11 / 2c95900e13ed5401aaba22176cca56f8c4851236) outdated

futures @ 0.1.29 / 0.3.28 (https://github.com/rust-lang/futures-rs/tree/0.3.28 / 1685f8bdabb8e7f007610f3494425aeabf28117d) multiple versions utilized (including outdated)

futures-rustls @ 0.21.1 / 0.22.1 (42afb9bdf6bf0a689ff51a65d93925880c085792) multiple versions utilized (including outdated)

futures-ticker @ 0.0.3 (https://github.com/antifuchs/futures-ticker/tree/v0.0.3 / f9c76c893cb17f7ec061d961d669f61325504a96)

hex @ 0.3.2 / 0.4.3 (https://github.com/KokaKiwi/rust-hex/tree/v0.3.2 / 19fd37137686c30058bd9d11d21590e726ffdf31) multiple versions utilized (including outdated)

lazy_static @ 1.4.0 (https://github.com/rust-lang-nursery/lazy-static.rs/tree/1.4.0 / 421669662b35fcb455f2902daed2e20bbbba79b6)

log @ 0.4.19 (https://github.com/rust-lang/log/tree/0.4.19 / 84ddc30ecc54e464b791dd591a5becc02f4fd06c) outdated

rand @ 0.6.5 / 0.7.3 / 0.8.4 (https://github.com/rust-random/rand/tree/0.8.4 / 8792268dfe57e49bb4518190bf4fe66176759a44) multiple versions utilized (including outdated)

regex @ 1.8.4 (https://github.com/rust-lang/regex/tree/1.8.4 / 5a34a39b72d85730065d3ffe4ce3715f2731e49a) outdated

rmp-serde @ 0.14.4 (https://github.com/3Hren/msgpack-rust/tree/rmp-serde/v0.14.4 / 00eeadfd60ae78cb0cbb0207c40efe9a2836697a) outdated

secp256k1 @ 0.20.3 / 0.24.3 (https://github.com/rust-bitcoin/rust-secp256k1/tree/secp256k1-0.24.3 / a1fb0386dcdf1fadd58c5384b2203a14fce56492) multiple versions utilized (including outdated)

serde @ 1.0.164 (https://github.com/serde-rs/serde/tree/v1.0.164 / 107018c6285e87f4cb265e1ef6c3d7931c1a15eb) outdated

serde_bytes @ 0.11.5 (https://github.com/serde-rs/bytes/tree/0.11.5 / 7f2e50869d87a50f45229d36740d9ad874ec9f4b) outdated

sha2 @ 0.8.2 / 0.9.9 / 0.10.7 (https://github.com/RustCrypto/hashes/tree/sha2-v0.10.7/sha2 / d8b088aece5659cecdce397ce48d6129b6c4c440) multiple versions utilized (including outdated)

smallvec @ 0.6.14 / 1.6.1 (https://github.com/servo/rust-smallvec/tree/v1.6.1 / 4e53e072808815ed6b847c77193c568ee076c29d) multiple versions utilized (including outdated)

syn @ 0.11.11 / 0.15.44 / 1.0.95 / 2.0.23 (https://github.com/dtolnay/syn/tree/2.0.23 / f3cd37a3fcd60f79fbf20581f91bc447e46d295a) multiple versions utilized (including outdated)

void @ 1.0.2 (https://github.com/reem/rust-void/tree/1.0.2 / a6e061227f47ba8798b7e828ed0ac4e25382eb15)

non-wasm specific deps:

instant @ 0.1.12 (https://github.com/sebcrozet/instant/tree/v0.1.12 / 9a486adeee119bdf7ecb85ba627b8112fb7eff86)
libp2p (https://github.com/KomodoPlatform/rust-libp2p.git) NOT REVIEWED
tokio @ 1.20 / 1.28.2 (https://github.com/tokio-rs/tokio/tree/tokio-1.28.2 / e87ff8a83a2edea1b23cf62269b8b31ef7f1df2d) multiple versions utilized (including outdated)

wasm specific deps:

instant @ 0.1.12 (https://github.com/sebcrozet/instant/tree/v0.1.12 / 9a486adeee119bdf7ecb85ba627b8112fb7eff86)
libp2p (https://github.com/KomodoPlatform/rust-libp2p.git) NOT REVIEWED

dev deps:

async-std @ 1.6.2 (https://github.com/async-rs/async-std/tree/caa76af745f986c2a12aac1b33b163b5a9633f05 / caa76af745f986c2a12aac1b33b163b5a9633f05 / NB: v1.6.2 tag was deleted from git) outdated
env_logger @ 0.9.3 (https://github.com/rust-cli/env_logger/tree/v0.9.3 / 3da11042a1f6a3b0808e52e534d1a39765bdb4ce) outdated

[shamardy]

@onur-ozkan can you please resolve conflicts so that we can merge this PR?

[CharlVS]

For anyone getting failed Android builds when trying to build the Android app bundle, it's a simple fix to upgrade minSdkVersion in your gradle file to 28 or newer. This will then use the compatible NDK version, so there's no need to manually override it in your gradle file.