chore(sequencer): add mempool benchmarks#1238
Merged
Conversation
steezeburger
added a commit
that referenced
this pull request
Jul 19, 2024
* main: (24 commits) chore: update `bytes` and `ics23` crates (#1279) fix(sequencer): improve and fix instrumentation (#1255) feature(charts): hermes chart fixes, bech32 updates, ibc bridge test (#1130) chore(cli): remove unused rollup cli code (#1275) chore(test): use a temporary file to not pollute the workspace (#1269) chore(sequencer): add mempool benchmarks (#1238) fix(bridge-withdrawer)!: fix nonce handling (#1215) feat(cli, bridge-withdrawer)!: share code between cli and service (#1270) feat(cli): add cmd to collect withdrawal events and submit as actions (#1261) fix(core, bridge, sequencer)!: dismabiguate return addresses (#1266) fix(withdrawer): support withdrawer address that differs from bridge address (#1262) (core, sequencer)!: generate serde traits impls for all protocol protobufs (#1260) fix(charts): add resources for sequencer/cometbft (#1254) chore(sequencer)!: add metrics (#1248) fix(sequencer-utils): fixes issue in `parse_blob` tests (#1243) feat(core, proto)!: make bridge unlock memo string (#1244) fix(conductor): don't panic during panic (#1252) feat(core)!: lowerCamelCase for protobuf json mapping (#1250) refactor(bridge-withdrawer)!: refactor startup to a separate subtask and remove balance check from startup (#1190) fix: rollup archive node configurations (#1249) ...
bharath-123
pushed a commit
that referenced
this pull request
Jul 25, 2024
## Summary Added benchmarks to `mempool` using [Divan](https://github.com/nvzqz/divan). ## Background There are further upcoming changes to mempool planned, and benchmarks will allow us to avoid committing poor performing code or blindly regressing on current performance. ## Changes - Added benchmarks. ## Testing These are tests. <details> <summary>Example of running <code>cargo bench -q -p astria-sequencer</code> on my Ryzen 7900X</summary> ``` Timer precision: 10 ns astria_sequencer fastest │ slowest │ median │ mean │ samples │ iters ╰─ mempool │ │ │ │ │ ╰─ benchmarks │ │ │ │ │ ├─ check_removed_comet_bft 44.82 µs │ 50 µs │ 45.23 µs │ 45.52 µs │ 100 │ 100 ├─ track_removal_comet_bft 44.98 µs │ 49.35 µs │ 45.33 µs │ 45.55 µs │ 100 │ 100 ├─ insert │ │ │ │ │ │ ├─ mempool_with_100000_txs 25.41 ms │ 40.04 ms │ 30.1 ms │ 30.87 ms │ 100 │ 100 │ ├─ mempool_with_10000_txs 2.228 ms │ 3.706 ms │ 2.58 ms │ 2.618 ms │ 100 │ 100 │ ├─ mempool_with_1000_txs 74.06 µs │ 101.5 µs │ 74.85 µs │ 75.82 µs │ 100 │ 100 │ ╰─ mempool_with_100_txs 12.5 µs │ 17.56 µs │ 12.76 µs │ 12.85 µs │ 100 │ 100 ├─ pending_nonce │ │ │ │ │ │ ├─ mempool_with_100000_txs 29.88 ms │ 38.8 ms │ 31.07 ms │ 31.78 ms │ 100 │ 100 │ ├─ mempool_with_10000_txs 2.099 ms │ 3.572 ms │ 2.73 ms │ 2.76 ms │ 100 │ 100 │ ├─ mempool_with_1000_txs 74.23 µs │ 105.4 µs │ 75.42 µs │ 76.26 µs │ 100 │ 100 │ ╰─ mempool_with_100_txs 11.66 µs │ 13.35 µs │ 11.93 µs │ 11.95 µs │ 100 │ 100 ├─ pop │ │ │ │ │ │ ├─ mempool_with_100000_txs 27.57 ms │ 37 ms │ 30.05 ms │ 31.3 ms │ 100 │ 100 │ ├─ mempool_with_10000_txs 1.971 ms │ 3.022 ms │ 2.614 ms │ 2.658 ms │ 100 │ 100 │ ├─ mempool_with_1000_txs 70.68 µs │ 96.14 µs │ 71.96 µs │ 72.71 µs │ 100 │ 100 │ ╰─ mempool_with_100_txs 11.3 µs │ 17.47 µs │ 11.76 µs │ 11.81 µs │ 100 │ 100 ├─ remove │ │ │ │ │ │ ├─ mempool_with_100000_txs 28.3 ms │ 34.65 ms │ 29.81 ms │ 30.56 ms │ 100 │ 100 │ ├─ mempool_with_10000_txs 2.093 ms │ 3.29 ms │ 2.643 ms │ 2.668 ms │ 100 │ 100 │ ├─ mempool_with_1000_txs 72.59 µs │ 96.33 µs │ 73.56 µs │ 74.09 µs │ 100 │ 100 │ ╰─ mempool_with_100_txs 11.69 µs │ 16.5 µs │ 11.94 µs │ 12.09 µs │ 100 │ 100 ╰─ run_maintenance │ │ │ │ │ ├─ mempool_with_100000_txs 47.28 ms │ 56.7 ms │ 50.59 ms │ 51.12 ms │ 100 │ 100 ├─ mempool_with_10000_txs 4.488 ms │ 6.047 ms │ 5.172 ms │ 5.176 ms │ 100 │ 100 ├─ mempool_with_1000_txs 310.5 µs │ 344 µs │ 315.4 µs │ 316.7 µs │ 100 │ 100 ╰─ mempool_with_100_txs 34.69 µs │ 40.81 µs │ 35.56 µs │ 35.74 µs │ 100 │ 100 ``` </details>
lobstergrindset
pushed a commit
that referenced
this pull request
Aug 13, 2024
## Summary This is effectively an update to the existing PR #1323. ## Background While reviewing #1323, it seemed more expedient to open a PR against it rather than adding many GH comments (this was discussed with @Lilyjjo). As always, there are more changes here than I initially expected. The main thrust of the changes was to try and use the type system to avoid certain classes of errors, and to make things a little more canonical where possible. ## Changes - At a high level, there are now separate types dealing with pending txs and parked txs: `PendingTransactionsForAccount`, `PendingTransactions`, `ParkedTransactionsForAccount` and `ParkedTransactions`. For the ones dealing with a single account, much of their logic is common, so a trait `TransactionsForAccount` was introduced to express that shared logic. For the ones dealing with collection of accounts' txs, a struct was introduced: `TransactionsContainer` generic over the type of the account-specific container `T: TransactionsForAccount`. This allowed for the shared logic to be expressed on the generic impl, with methods which are only relevant to pending being added to only that specialization, and similarly for the parked impl. - Extraneous fields were removed to avoid potential synchronization issues (e.g. `Mempool::all` and `TransactionContainer::all`) - Reduced scope of several of the locks made on the `RwLock`s - Various other smaller changes ## Testing - No additional tests were added, the existing tests were modified. - Benchmarks were run and reveal very promising results. These are a reflection of the changes made in the original PR rather than this one. Example output is included below, and for reference, the previous figures are available in the description of #1238. Summary is that for a mempool with 100 txs, we're seeing roughly a two times speed increase, and at 100,000 txs, we're seeing roughly a ten times increase. <details> <summary>Example of running <code>cargo bench -qp astria-sequencer</code> on my Ryzen 7900X</summary> ``` Timer precision: 10 ns astria_sequencer fastest │ slowest │ median │ mean │ samples │ iters ╰─ mempool │ │ │ │ │ ╰─ benchmarks │ │ │ │ │ ├─ check_removed_comet_bft 41.1 µs │ 47.7 µs │ 41.96 µs │ 42.81 µs │ 100 │ 100 ├─ builder_queue │ │ │ │ │ │ ├─ mempool_with_100000_txs 9.578 ms │ 13.59 ms │ 10.09 ms │ 10.17 ms │ 100 │ 100 │ ├─ mempool_with_10000_txs 613 µs │ 821.4 µs │ 734.7 µs │ 723.7 µs │ 100 │ 100 │ ├─ mempool_with_1000_txs 55.37 µs │ 68.09 µs │ 59.3 µs │ 59.44 µs │ 100 │ 100 │ ╰─ mempool_with_100_txs 9.377 µs │ 14.2 µs │ 9.913 µs │ 10.21 µs │ 100 │ 100 ├─ insert │ │ │ │ │ │ ├─ mempool_with_100000_txs 2.79 ms │ 3.327 ms │ 2.944 ms │ 2.968 ms │ 100 │ 100 │ ├─ mempool_with_10000_txs 137.9 µs │ 277.3 µs │ 201.4 µs │ 199.2 µs │ 100 │ 100 │ ├─ mempool_with_1000_txs 15.38 µs │ 24.64 µs │ 18.73 µs │ 18.78 µs │ 100 │ 100 │ ╰─ mempool_with_100_txs 7.303 µs │ 11.24 µs │ 7.579 µs │ 7.686 µs │ 100 │ 100 ├─ pending_nonce │ │ │ │ │ │ ├─ mempool_with_100000_txs 2.727 ms │ 3.524 ms │ 2.896 ms │ 2.92 ms │ 100 │ 100 │ ├─ mempool_with_10000_txs 124.4 µs │ 275.7 µs │ 198.2 µs │ 194.4 µs │ 100 │ 100 │ ├─ mempool_with_1000_txs 12.56 µs │ 22.14 µs │ 17.16 µs │ 16.97 µs │ 100 │ 100 │ ╰─ mempool_with_100_txs 6.001 µs │ 11.11 µs │ 6.176 µs │ 6.262 µs │ 100 │ 100 ├─ remove_tx_invalid │ │ │ │ │ │ ├─ mempool_with_100000_txs 2.76 ms │ 3.832 ms │ 2.945 ms │ 3.02 ms │ 100 │ 100 │ ├─ mempool_with_10000_txs 120.5 µs │ 288.1 µs │ 199.2 µs │ 201 µs │ 100 │ 100 │ ├─ mempool_with_1000_txs 14.76 µs │ 31.87 µs │ 18.22 µs │ 18.35 µs │ 100 │ 100 │ ╰─ mempool_with_100_txs 7.593 µs │ 18.72 µs │ 7.824 µs │ 8.123 µs │ 100 │ 100 ╰─ run_maintenance │ │ │ │ │ ├─ mempool_with_100000_txs 3.209 ms │ 3.877 ms │ 3.424 ms │ 3.456 ms │ 100 │ 100 ├─ mempool_with_10000_txs 423.5 µs │ 765.5 µs │ 497.7 µs │ 505.3 µs │ 100 │ 100 ├─ mempool_with_1000_txs 81 µs │ 125.8 µs │ 88.26 µs │ 88.84 µs │ 100 │ 100 ╰─ mempool_with_100_txs 14.14 µs │ 24.13 µs │ 14.64 µs │ 15.04 µs │ 100 │ 100 ``` </details>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Added benchmarks to
mempoolusing Divan.Background
There are further upcoming changes to mempool planned, and benchmarks will allow us to avoid committing poor performing code or blindly regressing on current performance.
Changes
Testing
These are tests.
Example of running
cargo bench -q -p astria-sequenceron my Ryzen 7900X