feat(wallet): Add a vertex history streamer to the wallet websocket API#1082
Merged
feat(wallet): Add a vertex history streamer to the wallet websocket API#1082
Conversation
b9937c8 to
6cd3fea
Compare
msbrogli
commented
Jul 11, 2024
6cd3fea to
4f71c51
Compare
msbrogli
commented
Jul 12, 2024
a8a77d0 to
b4f9209
Compare
msbrogli
commented
Jul 12, 2024
b4f9209 to
8ed7341
Compare
msbrogli
commented
Jul 12, 2024
8ed7341 to
ce31121
Compare
msbrogli
commented
Jul 12, 2024
38be174 to
2cb449e
Compare
r4mmer
reviewed
Jul 12, 2024
887c9f4 to
82c9605
Compare
238ec39 to
6a09d06
Compare
r4mmer
reviewed
Jul 19, 2024
f73ae2a to
5383196
Compare
jansegre
previously approved these changes
Jul 20, 2024
r4mmer
previously approved these changes
Jul 22, 2024
5383196 to
f42d005
Compare
52ae6e5 to
be56084
Compare
2f7ee1e to
631f2ca
Compare
glevco
approved these changes
Jul 24, 2024
r4mmer
approved these changes
Jul 24, 2024
This was referenced Jul 25, 2024
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Wallets might need to download their transaction history. The full node currently offers a polling API but this is too slow to both large wallets and high latency environments. This PR implements a streaming API that speeds up the download for wallets.
Streaming protocol
Client sends either
request:history:xpuborrequest:history:manual.Server open the streaming and follows the documented protocol (see the streamer docustring).
Performance
Testing with a wallet that has 1,060 addresses and 1,187 transactions with 480ms of latency between the wallet and the full node.
Download time
Total time (including a post-processing that occurs after the download):
Notice that the post-processing time is the same for all APIs (~10 seconds).
Acceptance Criteria
HathorAdminWebsocketFactory.handle_message(), moving it toHathorAdminWebsocketProtocol.HathorAdminWebsocketFactory._handle_ping(), moving it toHathorAdminWebsocketProtocol.HathorAdminWebsocketFactory.subscribe_address(), which subscribes the connection to receive real time updates for all vertices related to an address.request:history:xpub, which opens a streaming generating addresses from the xpub and stopping when the GAP limit is reached.request:history:manual, which opens a streaming from a list of addresses and stopping when the GAP limit is reached or the list of addresses ran out.request:history:stop, which stops the current streamer.request:history:manualto 5,000 addresses per connection.--disable-ws-history-streamingargument to therun_nodecli command.disable_history_streaming()to the factory that disables history streaming in all connections too.disable_history_streaming()to the protocol that stops the streaming if one is opened.Checklist
master, confirm this code is production-ready and can be included in future releases as soon as it gets merged