Skip to content

refactor(sync-v2): Modify TransactionStreamingClient to process vertices asynchronously#865

Merged
msbrogli merged 1 commit intomasterfrom
feat/sync-v2-async-tx-processing
Nov 10, 2023
Merged

refactor(sync-v2): Modify TransactionStreamingClient to process vertices asynchronously#865
msbrogli merged 1 commit intomasterfrom
feat/sync-v2-async-tx-processing

Conversation

@msbrogli
Copy link
Member

@msbrogli msbrogli commented Nov 10, 2023

Motivation

This PR enhances the on_block_complete() method, which initially processed transactions and added blocks to the DAG in a synchronous manner, thus occupying the event loop for the duration of this task. Due to the synchronous execution, the larger the transaction set, the longer the event loop was blocked, preventing the full node from attending to other critical operations like managing connections and processing API requests.

This PR transforms the method into an asynchronous operation. By doing so, after the addition of each transaction, control is yielded back to the event loop, allowing the full node to multitask effectively. This change is particularly beneficial during periods of heavy transaction processing, ensuring that node responsiveness and efficiency are maintained.

Acceptance Criteria

  1. Modify on_block_complete() to yield back to the event loop after calling on_new_tx() for each vertex.
  2. Remove the TransactionStreamingClient._buffer because it was not necessary anymore.
  3. Even if we receive a TRANSACTIONS-END message, we have to wait until all transactions have been processed before resolving the deferred.

Checklist

  • If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

@msbrogli msbrogli requested a review from glevco November 10, 2023 00:09
@msbrogli msbrogli self-assigned this Nov 10, 2023
@msbrogli msbrogli requested a review from jansegre as a code owner November 10, 2023 00:09
@msbrogli msbrogli force-pushed the feat/sync-v2-async-tx-processing branch from 5f93c92 to 4a7d448 Compare November 10, 2023 00:17
@codecov
Copy link

codecov bot commented Nov 10, 2023

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (382cc38) 85.42% compared to head (4a7d448) 85.31%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #865      +/-   ##
==========================================
- Coverage   85.42%   85.31%   -0.11%     
==========================================
  Files         282      282              
  Lines       22270    22308      +38     
  Branches     3367     3377      +10     
==========================================
+ Hits        19025    19033       +8     
- Misses       2583     2602      +19     
- Partials      662      673      +11     
Files Coverage Δ
hathor/p2p/sync_v2/agent.py 78.59% <100.00%> (-0.56%) ⬇️
hathor/p2p/sync_v2/transaction_streaming_client.py 85.50% <88.46%> (-4.69%) ⬇️

... and 6 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@msbrogli msbrogli force-pushed the feat/sync-v2-async-tx-processing branch from 4a7d448 to e85ad70 Compare November 10, 2023 15:52
@msbrogli msbrogli merged commit e85ad70 into master Nov 10, 2023
@msbrogli msbrogli deleted the feat/sync-v2-async-tx-processing branch November 10, 2023 15:52
@jansegre jansegre mentioned this pull request Nov 13, 2023
2 tasks
This was referenced Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants