Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PostgreSQL LISTEN/NOTIFY Subscriber #126

Open
roblaszczak opened this issue Sep 16, 2019 · 1 comment
Open

PostgreSQL LISTEN/NOTIFY Subscriber #126

roblaszczak opened this issue Sep 16, 2019 · 1 comment
Labels
discussion needed Further information is requested enhancement New feature or request help wanted Extra attention is needed M medium issue module: subscriber

Comments

@roblaszczak
Copy link
Member

roblaszczak commented Sep 16, 2019

The same as #5, but for PostgreSQL :)

This implementation should be added in https://github.com/ThreeDotsLabs/watermill-sql/ repository. It need to be compatible with already existing SQL Subscriber.

That means, that you should use SQL publisher for publishing messages, and binlog subscriber to read them.
We would, of course, keep already existing SQL based subscriber.

I'm not expert in PostgreSQL area, so any comments and ideas are welcome! 😉

It probably depends on #127.

@roblaszczak roblaszczak added enhancement New feature or request help wanted Extra attention is needed discussion needed Further information is requested module: subscriber M medium issue labels Sep 16, 2019
@breml
Copy link
Contributor

breml commented Feb 18, 2021

I looked into this topic myself for a similar idea (SQL backed event bus) and compared my ideas with the current implementation of watermill-sql. In my understanding, the benefits in the current implementation with LISTEN/NOTIFY wouldn't be this big, due to the guarantees currently given by the sql driver (guaranteed order of events, strictly once delivery). In my understanding the only thing that could be improved by LISTEN/NOTIFY in this case would be the polling would no longer be necessary, because the subscribers would be notified about new events.

So in the case of the benchmarks I would not expect an improvement at all, because there is no timeout/sleep/backoff as long as events are read and this is the case for the complete benchmark, because all the events are already present in the database before the read test is actually started (so the LISTEN/NOTIFY would again have no impact in this case).

In my opinion, LISTEN/NOTIFY would become interesting when some of the above mentioned guarantees would be softened, such that multiple readers in the same consumer group would actually be able to process events in parallel. For this to work, an additional table listing all message uuid per consumer group would be necessary, where the subscriber could read/update with SKIP LOOKED (or advisory locks).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion needed Further information is requested enhancement New feature or request help wanted Extra attention is needed M medium issue module: subscriber
Projects
None yet
Development

No branches or pull requests

2 participants