PubSub Research Review

[daviddias]

Hey!

Let's get this review done. As we mentioned in the last PubSub call (Mon 8, August), we've been collecting papers and reviewing them as the PubSub discussions progress. You can find the references through the link in the README -- https://github.com/libp2p/pubsub#pubsub-research-review-papers-books-talks-lectures-etc -- Unfortunately we can't link directly to the .pdfs, but they are rather easy to find through Google Scholar.

The collection has grown a little bit and we know it might look overwhelming at first (273 papers), but it is already a curated collection. It would be very good if everyone participating in the discussion could go through them as much as possible and take notes, there are a lot of papers with little secrets that can make the total difference when designing PubSub to scale for millions of users.

A good way to get started is through reading the survey -- XL peer-to-peer PubSub systems - http://dl.acm.org/citation.cfm?id=2543583 --, it will give you a very comprehensive overview.

If you find other articles, talks, lectures, papers, blog posts that are relevant to the PubSub discussion, please share them here or by summiting a PR to this repo.

Also, if you find something intriguing throughout the literature, be sure to post it here (or in a new issue) so that we can figure it out together.

[haadcode]

"PolderCast, a P2P topic-based pub/sub system that is fault-tolerant, robust, scalable and fast in terms of dissemination latency while attaining a low communication overhead"

https://www.duo.uio.no/bitstream/handle/10852/43117/1595-Setty-DUO-Thesis.pdf

[ghost]

Today we are proud to announce our first publication by HashiCorp Research, titled "Lifeguard: SWIM-ing with Situational Awareness". The paper details a number of novel improvements we have introduced to Serf, Consul, and Nomad to make their underlying gossip protocol more robust. Collectively called Lifeguard, these extensions reduce by 50x the false positives produced by the failure detector and allow us to detect true failures faster.

https://www.hashicorp.com/blog/making-gossip-more-robust-with-lifeguard/