Skip to content

chore(sequencer): refactor mempool#1092

Merged
Fraser999 merged 13 commits intomainfrom
fraser/refactor-mempool
May 27, 2024
Merged

chore(sequencer): refactor mempool#1092
Fraser999 merged 13 commits intomainfrom
fraser/refactor-mempool

Conversation

@Fraser999
Copy link
Contributor

Summary

The mempool was refactored to reduce the visibility, count and scope of acquired lock guards, and to be implemented in terms of a single collection rather than two which needed synchronized.

Background

The mempool has been implemented to be functionally correct, but can benefit from a refactor to improve safety, performance and complexity (by avoiding synchronizing two collections).

NOTE: This PR is based on top of #1073. Only the final two commits comprise the new work in this PR.

Changes

  • Having reduced the BasicMempool to only holding a single collection, there seemed little point to retaining it, so the Mempool now just holds the naked priority queue.
  • Changed from using a DoublePriorityQueue to a PriorityQueue since we don't need the extra functionality, and the latter is slightly more performant.
  • All mempool methods now take &self rather than &mut self.
  • The RwLock is no longer locked and unlocked in a tight loop in the insert_all and remove_all methods
  • TransactionPriority has been updated to only hold the nonce diff. There was a subtle bug in the previous impl since the derived PartialEq and Eq didn't correspond to the hand-rolled impls of PartialOrd and Ord. This has been fixed and further tests added.
  • SignedTransaction is internally held in an Arc now and passed in an Arc when executing by the app to reduce the number of clones. We could potentially look to extend this across more of the sequencer code for the same reason (not in this PR)
  • In the final commit, I changed how update_mempool_after_finalization was implemented, so that the mempool no longer has to expose a lock guard. I think it's generally much safer to not expose the lock or a guard on the lock in a non-private API. As well as moving the implementation inside mempool, I added a temporary in-mem cache of current account nonces, since it seems that there could be many txs all under the same account. As already noted, this method could be problematic due to holding the mempool lock for an extended duration, so this might reduce the risk.

Testing

Unit tests extended/added.

Related Issues

Closes #1083.

@github-actions github-actions bot added conductor pertaining to the astria-conductor crate proto pertaining to the Astria Protobuf spec sequencer pertaining to the astria-sequencer crate sequencer-relayer pertaining to the astria-sequencer-relayer crate labels May 21, 2024
}
}

fn priority(&self, current_account_nonce: u32) -> anyhow::Result<TransactionPriority> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if this returned an Option<TransactionPriority> instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if there's much of a win there. We'd just need to convert the None case to an Err in Mempool::insert unless we wanted to change its return type too?

@Fraser999 Fraser999 marked this pull request as ready for review May 23, 2024 18:51
@Fraser999 Fraser999 requested a review from a team as a code owner May 23, 2024 18:51
@Fraser999 Fraser999 requested a review from noot May 23, 2024 18:51
Copy link
Contributor

@noot noot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, this looks good to me!

@Fraser999 Fraser999 added this pull request to the merge queue May 27, 2024
Merged via the queue into main with commit 8067367 May 27, 2024
@Fraser999 Fraser999 deleted the fraser/refactor-mempool branch May 27, 2024 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conductor pertaining to the astria-conductor crate proto pertaining to the Astria Protobuf spec sequencer pertaining to the astria-sequencer crate sequencer-relayer pertaining to the astria-sequencer-relayer crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sequencer: refactor mempool internals

3 participants