Make ipfs-pubsub-room work with js-ipfs-api

[haadcode]

Starting with an IRC conversation:

02:28 <@haad> pedrot, daviddias: I'm facing a challenge bringing orbit-db usable with ipfs-api again: ipfs-pubsub-room. I was looking into it yesterday and
              realized (as we're using pubsub-room in orbitdb) that it ain't gonna work as it today with ipfs-api
02:29 <@haad> ...as we're using libp2p directly in pubsub-room
02:29 <@haad> so I'm thinking how to solve this and would appreciate your input
02:30 <@haad> been thinking 1) expose libp2p apis in http-api (prolly not a good idea and definitely a lot of work) 2) use primitives already available in
              http-api
02:31 <@haad> 2) makes a lot more sense to me in order to keep compatibility between js-ipfs ans js-ipfs-api
02:32 <@haad> so I ws thinking, what if we separated the direct connections from room and used PubSub to provide the same mechanism but with a "direct" topic
              between two peers
02:32 <@daviddias> haad I believe pedrot designed it that way so that there was actually a room and not a crowd
02:32 <@daviddias> PubSub is always floodsub
02:33 <@daviddias> but yeah, doing it right (1)) would be awesome but requires more than an evening of hacking
02:33 <@haad> something like (topic name): "/pubsub-direct/QmPeer1/QmPeer2"
02:34 <@haad> that ^ wouldn't be "private" by default in any way though, so I was thinking if we could add something to authenticate on floodsub level (perhaps
              just a "authenticate()" function passed to the subscription or smth, nothing fancy with keys)

Problem: ipfs-pubsub-room doesn't work with ipfs-api

Causes: We're using libp2p primitives in the direct connections, which are not available through the http-api on go-ipfs (and thus not available in js-ipfs-api)

Solution: ???

My strongest proposal atm is the one proposed ^ with a "direct" pubsub channel between two peers. I like it because it would decouple the direct-connection abstraction from room and it's something we can do using the primitives available in both implementations (regular pubsub primitives). The down side is that anyone can publish/subscribe to it without changes to floodsub, so it's definitely not private channel in any way. To address that, I was thinking we could add an argument to pubsub function that takes in a authenticate(peerId) function that the user provides when subscribing. The function would return true|false according to its own authentication rules, but in this case it would look something like: const authenticate = (peerId) => peerId && (peerId === this._receiverId || peerId === this._senderId) where _receiverId and _senderID are the IPFS IDs in the topic name ("/pubsub-direct/QmPeer1/QmPeer2").

I would love to hear other possible solutions as well as comments on the proposal ^.

[daviddias]

08:01 <@daviddias> the reason why it uses libp2p is so that all the comms between 2 peers are encrypted e2e
08:02 <@daviddias> if you use PubSub as you propose, then any peer can eavesdrop in other peers messages

[haadcode]

08:03 <@haad> daviddias: yup, in my proposal the encryption would have to happen on app/user-layer

[pgte]

@haadcode that's a great point, it really forbids us from using a non-local node with this package, but I think this is a limitation of ipfs-api and the HTTP API, not this package. Somehow, the IPFS API should allow to open raw private streams to specific nodes.

Also, I believe that encryption and authentication should not be a concern of this package, it should be addressed further up the stack, maximising separation of concerns.

As an alternative to fixing ipfs-api, I think there's nothing preventing you from building this mechanism on top of this package by "broadcasting" p2p messages in the way you described...

@haadcode @diasdavid thoughts?

[haadcode]

@pgte totally agreed. I think I'll go ahead and create the module as laid out ^ and see what that gets us (not sure when I get to work on it though). Ideally, as noted by all of us I think, we should have a way to hook into libp2p streams/connections directly from the API to get the benefits of libp2p.

[nezzard]

Any updates?

[pgte]

Nope. This depend on ipfs-api, which would have to be standardised and implemented in go-ipfs and js-ipfs.

[nezzard]

For now, we can use it only with js-ipfs?

[pgte]

Correct, but not in the remote API mode.

…

On 8 Apr 2018, 13:05 +0100, nezzard ***@***.***>, wrote: For now, we can use it only with js-ipfs? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[haadcode]

Update on OrbitDB side is that I ended up first taking out the libp2p functionality from ipfs-pubsub-room and created a module for 1-to-1 communication https://github.com/haadcode/ipfs-pubsub-1on1. Note that it's not a secure channel in the way ipfs-pubsub-room has: anyone can join the pubsub topic and listen in on it. This will change in the future, however.

Second, as ipfs-pubsub-room then only had the room notifications functionality left and we didn't want to use a fork, I created another module to do only that: https://github.com/haadcode/ipfs-pubsub-peer-monitor.

Essentially, those two modules have the same functionality as ipfs-pubsub-room but broken down per "abstraction", works with js-ipfs-api but also provide less security guarantees. I've created them in order to be able to keep moving on in OrbitDB, but the overall goal here is to be able to use ipfs-pubsub-room room in the long run once we have the libp2p connections available through js-ipfs-api.

@pgte you may want to check those modules and see what you think. Would be also more than happy to transfer the repos to shipyard if you feel they benefit the community and work done here.

[pgte]

@haadcode that's super, I think they may be very useful in some of upcoming work: I think I'm going to need direct communication, presence, but not broadcast.

I'd be super glad these repos come join the ipfs-shipyard org!

[haadcode]

@pgte tried to transfer the repos but got an error "You don't have the permission to create repositories on ipfs-shipyard".

Mind adding me to the org (would also love to keep admin/write access to the repos I'm transferring)?

[pgte]

It seams like I don't have enough permissions for that. @diasdavid an you help on this?

[daviddias]

Done :)

[haadcode]

@diasdavid @pgte thank you! ♥️

The repos have been transferred respectively at:

• https://github.com/ipfs-shipyard/ipfs-pubsub-1on1
• https://github.com/ipfs-shipyard/ipfs-pubsub-peer-monitor.

I believe I'm still the only one who can publish on npm and would be happy to add another maintainer for that, but you can also just give a shout and I'll publish when needed. We can cross that bridge when we get there.