Problem with the example in the libp2p-vpn folder

[aabbtree77]

Ubuntu 22.04 terminal 1:

./libp2p-vpn keygen -f peer1
peer ID: QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg

./libp2p-vpn keygen -f peer2
peer ID: QmYPh6xi528VbNjva4qxNYQ6f3GUZSFnWfPSoQ5ZfCDwgQ

sudo ./libp2p-vpn run -f peer1 -l /ip4/127.0.0.1/tcp/10000 -p /ip4/127.0.0.1/tcp/10001/p2p/QmYPh6xi528VbNjva4qxNYQ6f3GUZSFnWfPSoQ5ZfCDwgQ -t 10.0.0.1/24,10.0.0.2/24
[sudo] password for yoko: 
Trying to connect to /ip4/127.0.0.1/tcp/10001/p2p/QmYPh6xi528VbNjva4qxNYQ6f3GUZSFnWfPSoQ5ZfCDwgQ
[x] Connection succeeded!

The same machine, terminal 2:

sudo ./libp2p-vpn run -f peer2 -l /ip4/127.0.0.1/tcp/10001 -p /ip4/127.0.0.1/tcp/10000/p2p/QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg -t 10.0.0.2/24,10.0.0.1/24
[sudo] password for yoko: 
Trying to connect to /ip4/127.0.0.1/tcp/10000/p2p/QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg
[x] Connection succeeded!
2023-08-10T17:53:15.912+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: protocol not supported
2023-08-10T17:53:17.919+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: protocol not supported
2023-08-10T17:53:18.924+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: protocol not supported

After pressing ctrl+c on terminal 1, terminal 2 shows the following additional output:

2023-08-10T18:07:07.998+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:91	p2p->if: error reading frame size: %!w(*errors.errorString=&{stream reset})
2023-08-10T18:07:10.005+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:148	if->p2p: error writing frame size: stream reset
2023-08-10T18:07:11.012+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial tcp4 127.0.0.1:10000: connect: connection refused
2023-08-10T18:07:12.019+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial backoff
2023-08-10T18:07:13.012+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial backoff
2023-08-10T18:07:13.012+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial backoff
2023-08-10T18:07:13.022+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial backoff
2023-08-10T18:07:13.026+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial backoff
2023-08-10T18:07:13.650+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial backoff
2023-08-10T18:07:14.018+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial backoff
2023-08-10T18:07:14.019+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial backoff
2023-08-10T18:07:16.029+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial tcp4 127.0.0.1:10000: connect: connection refused
2023-08-10T18:07:16.029+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial backoff
2023-08-10T18:08:42.676+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial tcp4 127.0.0.1:10000: connect: connection refused
2023-08-10T18:09:10.012+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial tcp4 127.0.0.1:10000: connect: connection refused
2023-08-10T18:09:11.019+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial backoff
2023-08-10T18:09:12.028+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial backoff
2023-08-10T18:09:13.036+0300	ERROR	vpn	go-libp2p-vpn/vpn.go:133	if->p2p: error creating stream: failed to dial QmPcc3MZapc4LHkwR9H1ZYRh1qMgjiVM5jMVHotxyUkPjg:
  * [/ip4/127.0.0.1/tcp/10000] dial backoff

The errors are shown only in terminal 2. Please also notice that I had to prepend /24 to the virtual addresses as otherwise I would get some error about CIDR (when using -t 10.0.0.1,10.0.0.2 as indicated in the example readme).

I am also not sure what is the use of this example and how to set up the listening and peer addresses precisely in general. My typical case is two different machines on the global internet without static public IPs. Please provide a full command to run or indicate that it is not possible with this code because this is not very clear for me at the moment.

If I run the terminal 2 command on a different PC in the same LAN, there is no connection, both ends just wait, there are no errors displayed.

[balena]

Did you fix it with the last PR merged? If so, can I close this one?

[aabbtree77]

That PR was just to pass "go build", this is a different error that happens at runtime. There are more of these, but they seem to be non-blocking and act as low level warnings for all the practical purposes (they can be ignored).

Note 1

To clarify that vpn.go:133 error, I insert Println to check NewStream args:

   if vpn.stream == nil {
   		    s, err := host.NewStream(ctx, vpn.peer, vpn.Protocol())
   		    fmt.Println("vpn.peer:", vpn.peer, "vpn.Protocol():", vpn.Protocol())
   		    if err != nil {
   			    log.Errorf("if->p2p: error creating stream: %s", err)
   			    continue
   		    }

vpn.peer: QmdTct688mExQxs9nQ211vgNABmqn9H933rEBLM5AwreTd
vpn.Protocol(): /vpn/tun/1.0.0

So everything is alright here.

Note 2

The program gets correctly into the loop and keeps looping:

vpn.go:120

func (vpn *VPNService) Serve(ctx context.Context, host host.Host) {
	packet := make([]byte, maxMTU+ethernetMinimumSize)

	for {

I see these three read/write functions inside the loop, they are the ones that cause these occasional error messages:

vpn.go: 124
packetSize, err := vpn.Read(packet)
calls
/usr/local/go/src/os/file.go:115

vpn.go: 147
err = binary.Write(*vpn.stream, binary.LittleEndian, uint16(packetSize))
calls
/usr/local/go/src/encoding/binary/binary.go:338

vpn.go:155
_, err = (*vpn.stream).Write(packet[:packetSize])
calls
/root/go/pkg/mod/github.com/libp2p/[email protected]/p2p/host/basic/basic_host.go:1022
which then gets deeper into go-multistream functions.

packetSize is almost always 201, but it could become 48 when stream is reset or something else happens (did not go into details here), but packet is always []uint8 len=1564, cap=1564, I doubt this is useful, but I state it here just in case.

So the program loops perpetually executing these three packet read/writes with minor stream resets and checks between them.

The error appears once in a while when a packet is not read properly or there is some problem with a stream, there can be some other errors shown, but it in no way this disrupts the loop, the loop seems to restore itself and keeps executing these perpetually. When I debug, I can simulate the error by just pausing stepping, waiting some seconds, and then stepping again, the error gets printed on the other end as there is some expected timeout between these reads/writes deep inside libp2p. The program recovers, though it will not recover if I wait too long on the other end as there appear extra error messages like

failed to negotiate security protocol: context deadline exceeded

dial backoff

When this happens, things get totally busted as the stream gets repeatedly reset in vpn.go:131 host.NewStream(.. at each iteration.

Note 3

If someone is wondering how to run delve/gdlv on Ubuntu 22.04 with these sudo-requiring codes. Install delve and gdlv with go install at the latest versions, use "sudo -i", and then source .bashrc to get all these $DESKTOP like variables reloaded. Then gdlv debug run -f peer2 etc. The arguments passed to the debugged program come directly after "gdlv debug run".

Debugging will not be a cakewalk since the stuff needs to be executed on both ends in general, and there are goroutines. Timers also change code paths depending on how quickly you step through certain code paths. There will also be optimized functions, FFI, generics, go generate, debugger bugs, debugger's GUI bugs, some code bases like awl-tray also use weird self-executing code tricks to overcome sudo. All this makes debugging infeasible.

Note 4

My major use case is sshing into a remote machine with no static IP behind CGNAT, or sending messages between remote Go nodes, so I was expecting the example to do that, but it seems to be a more minimal low level tunneling example. It works fine as demonstrated with two terminals on the same machine, but it does not connect the terminals between different Ubuntu machines even in the same LAN.

Note 5

For those looking for more complete albeit still minimal examples of remote connectivity with go-libp2p, I have found and tested these two (they are terminal-based chat apps, compile and run without errors):

manishmeganathan/peerchat - MIT.
gbaranski/cryptogram - GPL3.

Note 6

This is not so relevant here, but I will mention a few big and interesting go-libp2p based projects: awl, Syncthing, Berty... There is a comment on Berty's Google Play page that states the app has been blocked in Iran, so go-libp2p has already made some impact. These tools change the lives of people. functionland provides some links for the use of go-libp2p in the context of decentralized file storage.

[balena]

Protocol? The underlying transport is provided by libp2p and the PoC simply opens a bidirectional stream so nodes send datagrams over it, so protocol is simply size + data. As PoC the objective was to demonstrate how someone could implement a VPN using libp2p... for fun. It is not intended for commercial use neither serious use except you know what you're doing very well.

There is no much sense to implement even another protocol on top of libp2p because privacy, authenticity and integrity is already provided by it.

And yes, it binds to actual TUN/TAP interfaces through water, so it tunnels IP/Ethernet packets directly (depends if you use TUN or TAP).

Regarding this TUN/TAP part, the water library, you basically open up a network interface where your app receives every packet and can decide what to send back, so you can pretty much build virtual remote links this way. On Linux you generally use iptables or equivalent to NAT.

Regarding government blockings, it's a mouse chase game...

What problem you want to solve exactly? Maybe I can help you better knowing what you want to achieve.

[aabbtree77]

Thank you for clarifying these things. I have made a few minor edits in my big previous comment, but I could have just deleted it all as the errors have turned out to be non-errors. I will leave it as it is because someone might find something useful here.