Feature request - incoming transaction notifications

[szollo]

It would be amazing to have support for desktop notifications of incoming transactions, so that the user is informed when there's new incoming bitcoin without needing to refresh the browser, or indeed even look at the web interface - this is pretty much the only feature preventing me to fully switch from Electrum at this point.

Taking a cursory look this could be achieved by using ZMQ subscriptions, however as it's not plug-and-play it would probably have to be an advanced setting, as Bitcoin Core default config doesn't enable it (though it's compiled into the official binaries afaik).

The other challenges I can think of, off the top of my head:

• Having cross-platform desktop notification (perhaps just browser notifications instead of OS native would suffice?)
• Extra configuration steps if Specter is connecting to the node via Tor, seeing as the ZMQ port would also need to be published as a separate hidden service
• There's probably a client / frontend websocket component to develop?

Thanks for the great work, Specter is awesome.

[relativisticelectron]

I thought the enhancement is a good idea and developed a proof-of-concept for it: https://github.com/cryptoadvance/specter-desktop/compare/master...relativisticelectron:notification?expand=1

It uses https://developer.mozilla.org/en-US/docs/Web/API/Notifications_API/Using_the_Notifications_API
There is a simple javascript checker (setInterval), checking for updated transactions.

[relativisticelectron]

@moneymanolis : Do you think this web-push notification would be useful for many users? Or are most users running bitcoin-qt locally, and have therefore the notifications from bitcoin-qt?
@szollo : Is this web-push notification what you had in mind?

If there is demand for this feature I can also check out https://bitcoindev.network/accessing-bitcoins-zeromq-interface/ . Something like js <--> python (currently via setInterval and GET), and python <--> Bitcoin-core (zqm, currently rpc)

[szollo]

That's exactly what I had in mind @relativisticelectron - looks great!

[moneymanolis]

@relativisticelectron looks cool! As for your question about bitcoin-qt, I think nobody uses this, users have usually different variations of bitcoind instances running, either locally or on a remote node.

Tagging @k9ert for architectural input.

Sth. that directly caught my eye from the MDN documentation:

Don't we have a user group that struggles with not being able to connect via HTTPS with their (remote) node? I think Umbrel, no?

[relativisticelectron]

Don't we have a user group that struggles with not being able to connect via HTTPS with their (remote) node? I think Umbrel, no?

Yes, I have no idea how to deal with this. It seems https is not coming to umbrel any time soon: getumbrel/umbrel-os#251

It works on localhost though...

[k9ert]

Wow, this really looks cool. Somehow overlooked this issue. I think the zeromq thing should be more in focus than the frontend-implementation as otherwise, you just create unnecessary core-calls which might cause unnecessary loads on small raspis.

[relativisticelectron]

@k9ert : Thanks for the hint. A zmq client (proof of concept) turned out to be super easy in python. I added zmq such that rpc calls for notifications are ONLY done if the zmq has gotten a message: https://github.com/cryptoadvance/specter-desktop/compare/master...relativisticelectron:notification?expand=1

1. The problem with zmq is, that the user has to enable it in bitcoin.conf. Is there a way to change the bitcoin.conf of the bitcoin node that is bundled with specter?
2. Is there a way to make the javascript <--> python connection more efficient? How to listen to a server side event with javascript? I have now (in this branch) instant push notifications using pywebpush, though it only works on Chome/Chromium... Firefox gives the error
   . So perhaps the simple setInterval checker is better, because it works in Firefox too...

[k9ert]

Wow, this really looks promising and also easier than i thought. When i get that right, you don't clear the buffer to the server-side data-structures directly but just create the notification on the client-side (via continuous pulling and maybe later via web-push) and the then the user is encouraged to make a normal request requests the data. We probably need to also put some logic in for the case: It should profit also via not asking core that often anymore but all of that can wait for now.

Not sure how complex the web-push is but maybe we can separate the two things? I'd probably hesitate to merge web-push if it's not working on firefox but who cares for cheap and open small polls?!

I just had a look in the internal node implementation. Indeed, we have a bitcoin.conf. If we modify that one, we would need a SpecterMigration. But i don't think it's necessary. I don't think we should have created a bitcoin.conf in the first place and another alternative is to "add" the zmq-config via the commandline here:
https://github.com/cryptoadvance/specter-desktop/blob/master/src/cryptoadvance/specter/process_controller/node_controller.py#L333

Can you try to add the necessary ZMQ-stuff into the commandline? Maybe create an extra config.py config value which makes it possible to deactivate ZMQ. IMHO, the commandline options come on top of the bitcoin.conf.

[relativisticelectron]

Thanks for the feedback!

• Great that you think that the simple javascript setInterval checker is sufficient. The checker checks if any messages are in the zmq buffer and creates the Notification.
• Sidenote: I first thought pywebpush would work also in firefox, but after I got it working was disappointed that it only works in Chrome. Additionally I don't like the idea of sending the (encrypted) notifications through the google cloud -- so I move the pywebpush code onto a different (trash) branch.
• Yes I will put the zmq it in the commandline options and test it soon.
• new gif showing messages after a tx is confirmed: https://user-images.githubusercontent.com/60378539/164282034-7ab70f3b-0394-422a-a2ba-4832b374849c.gif
• If specter is not running on localhost the https requirement blocks the Notification request:
  

[relativisticelectron]

Javascript Notifications

If the permission cannot be granted (because of https missing), and is not denied I can at least show javascript notifications:

@szollo : Would these Javascript notifications (in the browser window) be helpful if the Notifications_API (docs) is not available?

[szollo]

That looks great @relativisticelectron!

@szollo : Would these Javascript notifications (in the browser window) be helpful if the Notifications_API (docs) is not available?

For the use case I had in mind I'd prefer it was native to the OS (ie. macOS or Windows notifications) or browser, as more often than not I don't view the active window, especially when travelling and using one screen.

That being said, it's great to have the choice - so if I wanted to have that feature I'd have to make sure I run Specter as an HTTPS service with a trusted certificate?

[relativisticelectron]

For the use case I had in mind I'd prefer it was native to the OS (ie. macOS or Windows notifications) or browser, as more often than not I don't view the active window, especially when travelling and using one screen.

Sure, that is best. Without https this Notifications_API (docs) (remotely) is just not possible. I still have to investigate if the push service (google or mozilla) works without https, but it would involve a cloud server that forwards encrypted messages.
TODO:

• test pywebpush branch via remote
  Specter in Chromium. --> It doesn't because https is missing.
• if it works (then it is the way forward), try the mozilla cloud server
• Does pywebpush branch work with tor-browser ? --> It doesn't
• Does Notifications_API (docs) work with tor-browser? Yes it does

That being said, it's great to have the choice - so if I wanted to have that feature I'd have to make sure I run Specter as an HTTPS service with a trusted certificate?

1. Notifications_API (docs) on Localhost doesn't require a certificate.
2. Yes for remote specter, Notifications_API (docs) does require a certificate. But since this is probably not available in most node packages, maybe javascript notifications are better than no notifications at all? Maybe the website tab title changing from to "(1) Specter Desktop"?
3. Notifications_API (docs) seems to work well in the tor browser :-)

Bottom line

• Notifications_API (docs) for specter on localhost http or tor browser
• Javascript Notifications for remote http specter
• Basic principle: javascript setInterval checks for new tx in buffered_zmq_messages <--> python zmq client stores zmq tx messages in buffered_zmq_messages <-- bitcoin zmq server broadcasts zmq tx messages

I that OK for everyone @k9ert , @moneymanolis @szollo ? Should I (slowly) develop this proof-of-concept into a PR?

[k9ert]

Again missed your comment even though in the meantime i'm indeed using the github-task/notification system.

I that OK for everyone @k9ert , @moneymanolis @szollo ? Should I (slowly) develop this [proof-of-concept]
(https://github.com/relativisticelectron/specter-desktop/tree/notification) into a PR?

I think this looks very promising. Maybe we can somehow combine that with #1695 ? On the browser-side, I'd focus for now on the Javascript Notifications and defer the notification-API to the very end. So imho the order should be:

• basic notification system (maybe with some pages to read/archive)
• ZMQ connection integrating with the notification-system
• Javascript Notifications
• Notifications API

Thank you so much, you're doing a great work!