ipfs p2p feedback thread

[whyrusleeping]

For everyone trying out the ipfs p2p command, please add any and all feedback here, this includes (but is not limited to) bugs, opinions, interface critiques, potential usecases, and feature requests.

To enable the command, you will need to run:
ipfs config --json Experimental.Libp2pStreamMounting true
And restart your daemon.

Basic usage of ipfs p2p is as follows:

First, open up a p2p listener on one ipfs node: `p2p listener open p2p-test /ip4/127.0.0.1/tcp/10101` This will create a libp2p protocol handler for a protocol named `p2p-test` and then open up a tcp socket locally on port 10101. Your application can now connect to that port to interact with the service.

On the other ipfs node, connect to the listener on node A
ipfsi 1 p2p stream dial $NODE_A_PEERID p2p-test /ip4/127.0.0.1/tcp/10102
Now the ipfs nodes are connected to eachother. This command also created another tcp socket for the other end of your application to connect to. Once the other end connects to that socket, both sides of your application should be able to communicate, with their traffic being routed through libp2p.

The easiest way to try this out is with netcat.

by @magik6k:
The feature is getting refactored in some breaking ways. For updated guide see https://github.com/ipfs/go-ipfs/blob/4bafbf1cdef7d6d0041154ee03efb02f2ef0685b/docs/experimental-features.md#ipfs-p2p.

What changed: #3994 (comment)

[magik6k]

What I noticed missing from one of the earlier PRs is the ability to tell to which peer connected to a listener. Earlier PRs had a switch that caused ipfs to send <PeerID>\n at the beginning of each incoming stream. This should be pretty simple to implement.

[ReisenB]

Is this planned to be accessible through the browser with js-ipfs-api?

[whyrusleeping]

Unfortunately no, you could set up the listeners from the browser, but you couldn't actually connect to it since you can't dial TCP connections there. Unless... @magik6k could you specify a /ws multiaddr for the listeners?

…

On Wed, Jun 28, 2017, 05:22 ReisenB ***@***.***> wrote: Is this planned to be accessible through the browser with js-ipfs-api? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3994 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABL4HDTg_Rqp6Rg4nUp69-AhryuOL3O2ks5sIhs8gaJpZM4N-zsb> .

[magik6k]

I'd need to look into how websockets are currently handled in go-ipfs. For dialing this should be relatively simple to do, though listen mode would probably have to use some sort of stream muxer that has js implementation. This should fit quite well into the existing code.

[jdgcs]

No opening port 10101, is that right?

liu@nas:~/ipfs/go-ipfs % ./ipfs p2p listener open p2p-test /ip4/127.0.0.1/tcp/10101

{"Address":"/ip4/127.0.0.1/tcp/10101","Protocol":"/p2p/p2p-test"}

liu@nas:~/ipfs/go-ipfs % ./ipfs p2p listener ls

/ip4/127.0.0.1/tcp/10101 /p2p/p2p-test

liu@nas:~/ipfs/go-ipfs % netstat -an | grep 10101

liu@nas:~/ipfs/go-ipfs %

[magik6k]

This creates listening libp2p handler and when p2p connection is incoming it(ipfs daemon) then opens a tcp connection to a given address and proxies the data. This seems counterintutive at first, but it allows for transparent connection proxying over libp2p

[origama]

It would be nice to be able to set the listeners in the config file, so that the daemon could create them at booting time. It makes sense especially if you run IPFS as an init/systemd/docker unit.

[magik6k]

I don't think using config for that is a good idea. What may work is extracting whole ipfs p2p into a separate program as it doesn't really fit ipfs (imo), this program could then talk to ipfs daemon and reuse its libp2p instance or be standalone.

[origama]

And adding up to my previous comment, I noticed a different behavior between the listener and the streamer.

Considering the case in which you start a listener for the protocol ipfs-test on peerA attached to the local port 80, and then you start a streamer on peerB to connect to peerA[ipfs-test] and listening on his local port 8080.

Now, with this setup, no matter if the streamer will disconnect or if the service on port 80 on peerA will die, the listener will still be up and running.

The streamer instead dies after whatever client running on peerB disconnects from port 8080.

Maybe it's expected, but it would be nice to have the streamer keep listening for more than one connection, or at least having a parameter (like a sort of nc -k), and maybe a config entry for this too.

[magik6k]

#4929 includes api-breaking changes, it will likely be released with the new version of go-ipfs.

EDIT: This changed a bit while the PR was open, this is the current version:

For updated how-to see https://github.com/ipfs/go-ipfs/blob/b84c386124441c1ecf261d7c4dbbc72112632299/docs/experimental-features.md#ipfs-p2p

What changes:

• internal changes:
  • listener is now referred to as 'remote listener'
  • stream(-er) is now referred to as 'local listener'
• we don't stop listening for local connections after accepting a single connection.
• ipfs p2p stream ls output now contains more useful output, first address is always the initiator address.
• ipfs p2p listener ls is now ipfs p2p ls
• ipfs p2p listener close is now ipfs p2p close
• Protocol names have to be prefixed with /x/ and are now just passed to libp2p as handler name. Previous version did this 'under the hood' and with /p2p/ prefix. There is a --allow-custom-protocol flag which allows you to use any libp2p protocol name.
• ipfs p2p listener open and ipfs p2p stream dial moved to ipfs p2p [listen/forward]
  • ipfs p2p forward /x/[protocolName] [listen multiaddr] [target multiaddr]
  • For listener open, instead of:
    • ipfs p2p listener open p2p-test /ip4/127.0.0.1/tcp/10101
    • you now do
    • ipfs p2p listen /x/p2p-test /ip4/127.0.0.1/tcp/10101
  • For stream dial instead of:
    • ipfs p2p stream dial $NODE_A_PEERID p2p-test /ip4/127.0.0.1/tcp/10102
    • you now do
    • ipfs p2p forward /x/p2p-test /ip4/127.0.0.1/tcp/10102 /ipfs/$NODE_A_PEERID

[DavidHuie]

@magik6k what's stopping ipfs p2p from supporting UDP?

[magik6k]

With #4929 you will be able to pass streams to udp services. You still won't be able to listen for udp connections as local listener uses go-multiaddr-net.Listen directly, and it only supports listening on tcp now.

There are some problems with supporting udp properly:

• Need to track connections if we want 2-way communication (this isn't that hard, maybe slightly harder if one wants this to be any fast)
• Libp2p operates on streams, udp is message oriented. There are 2 ways to solve this:
  • wrap udp packets into messages (basically size+data piped into stream)
    • IMO the wrong way to do this
  • give libp2p the concept of messages
    • There are some plans for message oriented (+unreliable) transports/muxers
      • Can't find any specific issue
    • This would be ideal for udp
    • Also, useful in other parts of libp2p.

[DavidHuie]

@magik6k Is that something that you guys are open to having the community implement?

I'm designing a system that uses IPFS to create a peer-to-peer VPN. All of the VPN protocols I'm considering use UDP since TCP is implemented at a higher level networking stack.

[DavidHuie]

Also, net.Conn in Go implements UDP connections as streams, so it wouldn't be a terrible way to implement that...

[whyrusleeping]

@DavidHuie We're definitely open to having the community help there. It's something that will be tricky to get right, adding a message oriented layer to libp2p won't be trivial. If you want to get the ball rolling, open an issue in libp2p/libp2p about it, describe use cases, and tag people. We can start to brainstorm thoughts there.

I know that @lgierth was planning on working on that soon, he's out on vacation for another week or so. When he gets back maybe you two can work on pushing that?

[Stebalien]

The refactor has been merged (massive thanks to @magik6k for keeping with this). Go pull master and try it out.

[Mikaela]

Is it possible (or planned) to use this feature as a reverse proxy?

I am thinking of FruitieX/teleirc#262 which is a Telegram/IRC relaybot and it includes HTTP server for Telegram file uploads / multimedia. I guess using it that way would lose a lot of advatages of the bot directly supporting IPFS though, unless the reverse proxy could add content by itself and the gateways cached or deduplicated it behind the scenes even if the filename kept changing.

[magik6k]

It is possible to use this as a reverse proxy, but it will just pass the stream without really caring what's inside.

We also have a separate http proxy feature - https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#p2p-http-proxy, it doesn't handle caching either, but (with some special header magic) it should be possible to implementet this.

[bojidar-bg]

Not sure if this issue is still the place for bugs and feedback on ipfs p2p, but using it for a project, I was surprised that half-closing a forwarded TCP socket closes the whole stream. Don't have a minimal example lying around anymore, but from what I managed to intuit, this piece of code tears down the whole connection as soon as either half of the socket is closed, e.g. with Go's TCPConn.CloseWrite.

Also, from experimentation, it seems that p2p does not support local-to-local connections (when passing the local peer's address as a target address), which makes quick testing setups needlessly convoluted.