Private Networks - First Implementation

[Kubuxu]

As private networks are adoption blockers for and few important cases were raised for them, I would like to describe to you our first take on Private Networks.

For the first implementation only Pre-shared Key functionality will be available, given the timeframe and complexity Public Key Infrastructure would be just too complex and needs more talk and preparation. PSK based approach will unblock some of the users and allow them to start using IPFS.

The key will be just one per instance of libp2p Swarm, and thus one per IPFS daemon. This clears up corner cases of "what if the node connects to private and public network at the same time".

We wanted to make it key-per-listen-address but it would be too complex.

---

The crypto was changed but it will be symmetric encryption.

---

The PSK will be stored inside swarm.key (if there is better place for it, please comment) file in repo with text based multicoded, occupying /key/swarm/psk/1.0.0/ as a name. Following it is a second text based multicodec defining base encoding of the following data, thus allowing the keys to be either typed easily manually (base16, base58), transferred over clipboard and shell without problems (base64) or can be also purely binary if you wish so.

Example:

 > cat $IPFS_PATH/swarm.key
/key/swarm/psk/1.0.0/
/b16/
FFFF

This is of course very weak key but it works as an example.

---

This feature is planned to be done at the end of the November, but it might be earlier. It is already in progress, testing it will take a while.

Part of: #1633
Resolves partially: ipfs/notes#146
Relates to: #1633

[Kubuxu]

cc @whyrusleeping

[whyrusleeping]

cc @jbenet for review. This is the initial implementation that @Kubuxu and I have been discussing.

[cpacia]

but is secio specific, when we move away from it we will have to rethink it.

What is the plan to move away from secio?

[Kubuxu]

@cpacia to keep things on topic in this issue I will link an issue from this comment when we have clean writeup about this.

[jbenet]

@cpacia the gist is we'll support TLS 1.3 in the future.

[jbenet]

@Kubuxu:

I don't think we should be doing this inside secio. We've discussed elsewhere that this feature should to be a wrapper around the entire connection, so that all bytes (starting from the first multistream) are encrypted, and so that it works around different other secure transports. We can adjust our viewpoint on this, but let's investigate other possibilities first.

Relevant here will be to measure the overhead of symmetric encryption (AES-CTR, AES-GCM, or ChaCha/Poly1305).

---

from previous discussions:

Requirements:

• every byte should be encrypted with the key. transport should not show it is ipfs. (i.e. encrypt even the first multistream handshakes). This gives private networks a better chance through censored networks.
• solve once, not per authenticated transport. we wont have the ability to modify all transports.
• use symmetric keys, allow choice of AES or ChaCha
• non-network nodes must not get stuck trying to dial us over and over (which could happen if the multistream handshake works but the auth channel fails inexplicably)
• ideally 0 RTT - it should not add any RTTs
• ideally no MAC overhead (non auth is ok in wrappers, as long as we auth in internal cipher)

Some constructions we've thrown around before: ipfs/notes#177

[Kubuxu]

I have sent a new XSalsa20 based suggestion in the ipfs/notes#177.

[Kubuxu]

I have benchmarked AES-CBC and XSalsa20 (including nonce change per round) and XSalsa20 is clear winner here:

On i7-6800k:
BenchmarkEncryptAesCbc-12           500000          3541 ns/op     406.57 MB/s
BenchmarkDescryptAesCbc-12          500000          3007 ns/op     478.76 MB/s
BenchmarkXSalsa20-12               1000000          1670 ns/op     854.37 MB/s

On RPi 3:
BenchmarkEncryptAesCbc-4           10000        203769 ns/op       7.07 MB/s
BenchmarkDescryptAesCbc-4          10000        202064 ns/op       7.13 MB/s
BenchmarkXSalsa20-4                10000        107729 ns/op      13.25 MB/s

The schema I described here, should fit all goals of this layer: high performance, nonce uniqueness (192bit), no re-keying, no per-packet overhead and it is 0-RTT.

[Kubuxu]

This is right now blocked on @jbenet finishing review on pnet code.

[mbialon]

I've tried to build the feat/pnet branch but I got the following error.

installing package: go-libp2p-pnet-0.1.1
  - QmRicHyjxHoiB6bY8bT8k9GoiiDKUS2DU5J23g7DPcZsXb not found locally, fetching into /Users/marcin/go/src/gx/ipfs/QmRicHyjxHoiB6bY8bT8k9GoiiDKUS2DU5J23g7DPcZsXb
  - QmbyJsuKfxWmcCMfHEGvqofjV8U3QP8ZQxDeAy1GmY9dW1 not found locally, fetching into /Users/marcin/go/src/gx/ipfs/QmbyJsuKfxWmcCMfHEGvqofjV8U3QP8ZQxDeAy1GmY9dW1
  - QmP6H1kAKEHohE2U4GE4PiiRtXp37L5Bxfwi4EvxV7Jn5K not found locally, fetching into /Users/marcin/go/src/gx/ipfs/QmP6H1kAKEHohE2U4GE4PiiRtXp37L5Bxfwi4EvxV7Jn5K
  - fetching QmRicHyjxHoiB6bY8bT8k9GoiiDKUS2DU5J23g7DPcZsXb via ipfs api
  - fetching QmbyJsuKfxWmcCMfHEGvqofjV8U3QP8ZQxDeAy1GmY9dW1 via ipfs api
  - fetching QmP6H1kAKEHohE2U4GE4PiiRtXp37L5Bxfwi4EvxV7Jn5K via ipfs api
  - fetch finished in 1.517754009s
  - fetch QmbyJsuKfxWmcCMfHEGvqofjV8U3QP8ZQxDeAy1GmY9dW1 complete!
[1 / 5] fetched dep: go-libp2p-dummy-conn
  - fetch finished in 1.983279134s
  - fetch QmP6H1kAKEHohE2U4GE4PiiRtXp37L5Bxfwi4EvxV7Jn5K complete!
[2 / 5] fetched dep: go-multicodec


ipfs-shell: warning! unhandled response encoding: text/htmlERROR: from shell.Get(): &shell.Error{Command:"get", Message:"unknown ipfs-shell error encoding: \"text/html\" - \"<html>\\r\\n<head><title>504 Gateway Time-out</title></head>\\r\\n<body bgcolor=\\\"white\\\">\\r\\n<center><h1>504 Gateway Time-out</h1></center>\\r\\n<hr><center>nginx/1.10.1</center>\\r\\n</body>\\r\\n</html>\\r\\n\"", Code:0}
retrying fetch QmRicHyjxHoiB6bY8bT8k9GoiiDKUS2DU5J23g7DPcZsXb after a second...

Would it be possible to build and test the private networks feature?

[Kubuxu]

I should have new version out this week.

[vivekar1234]

When can we get the latest one...please can you help as we have POC pending

[Kubuxu]

#3697 build available at https://ci.ipfs.team/job/go-ipfs/job/PR-3697/

To generate swarm key use: https://github.com/Kubuxu/go-ipfs-swarm-key-gen

[hmeine]

I would like to evaluate this preview version, but the above ci.ipfs.team-link is no longer valid. Can you provide a new link (and/or maybe two sentences on how one could find an up-to-date link from that site)? Thanks in advance!

[Kubuxu]

It is already released, every version since 0.4.7 supports it and here is more about it: #3397 (comment)

[hmeine]

Wow! I missed that, although I watched multiple related issues. Thanks!

[hmeine]

OK, so you wanted to get feedback; I just found the first issue: I wanted to use the webui, but the browser would not load the page. At some point (maybe only when stopping the daemon) I got an error message "ipfs resolve -r /ipfs/QmPhnvn747LqwPYMJmQVorMaGbMSgA7mRRoyyZYz3DoZRQ/: Failed to get block for QmPhnvn747LqwPYMJmQVorMaGbMSgA7mRRoyyZYz3DoZRQ", which may be because the webui tries to load the globe(?), but I am not connected to the global swarm.

• You might say this is by design, and one needs to make sure the data files are available in the local swarm. In that case, I would suggest to add a hint how to efficiently load these files into the local datastore to docs/experimental-features.md.
• I wonder if it makes sense to add the possibility to specify a second daemon as fall-back for missing blocks? That could allow "read-only" access to the global swarm, while still preventing local files to be seeded to nodes without the PSK.

[Kubuxu]

Yes, webui is mostly fetched on need basis, which means if you are in private network you won't be able to fetch it.

What you could do is start second fresh node, fetch the webui and then add it to private network.

wonder if it makes sense to add the possibility to specify a second daemon as fall-back for missing blocks?

This would just introduce complexity that would have to be supported in future.
I would prefer, instead of doing that, to move on the PKI based private networks with content policies/capabilities but I am afraid it is quite a ways off.

[kamaci]

@Kubuxu I have some questions on this feature.

1. Why key generation is not a part of ipfs commands and can be generated via user i.e. https://github.com/Kubuxu/go-ipfs-swarm-key-gen

2. Is there any solution to encrypt the data send over ipfs nodes to prevent man in the middle attacks?

[bonedaddy]

@kamaci I believe the data is encrpyted via the PSK

[Kubuxu]

@kamaci IPFS by default is using encryption that prevents MitM attacks, it continues to use this encryption when running Private Networks.