perf(sequencer): add benchmark for prepare_proposal (ENG-660)#1337
Merged
perf(sequencer): add benchmark for prepare_proposal (ENG-660)#1337
Conversation
noot
reviewed
Aug 20, 2024
noot
reviewed
Aug 20, 2024
| .unwrap(); | ||
| let mut fixture = runtime.block_on(async { Fixture::new().await }); | ||
| bencher | ||
| .with_inputs(|| BlockSizeConstraints::new(22_019_254).unwrap()) |
Contributor
Author
There was a problem hiding this comment.
It was taken from the actual value seen in prepare_proposal.max_tx_bytes when handling prepare_proposal during stress testing using spamoor. I'll add a comment.
noot
approved these changes
Aug 20, 2024
steezeburger
added a commit
that referenced
this pull request
Aug 22, 2024
* main: refactor(core, proto)!: define app genesis state in proto (#1346) fix(sequencer): bump penumbra dep to fix ibc state access bug (#1389) feat(conductor)!: support disabled celestia auth (#1372) fix(sequencer)!: fix block fees (#1343) perf(sequencer): add benchmark for prepare_proposal (ENG-660) (#1337) fix(proto): fix import name mismatch (#1380) fix(ci): enable bridge withdrawer building with tag (#1374) feat(sequencer): rewrite memool to have per-account transaction storage and maintenance (#1323) refactor(core, sequencer)!: require that bridge unlock address always be set (#1339) fix(sequencer)!: take funds from bridge in ics20 withdrawals (#1344) fix(sequencer)!: fix TOCTOU issues by merging check and execution (#1332) fix: abci error code (#1280) refactor(core): shorten `try_from_block_info_and_data()` (#1371) fix(relayer): change `reqwest` for `isahc` in relayer blackbox tests (ENG-699) (#1366) fix(conductor): update for celestia-node v0.15.0 (#1367) Chore: Upgrade celestia-node to v0.14.1 (#1360) chore(charts): fix charts production templates (#1359) chore(core, proto): migrate byte slices from Vec to Bytes (#1319)
Fraser999
added a commit
to Fraser999/astria
that referenced
this pull request
Aug 23, 2024
…org#1337) ## Summary Addition of a new benchmark target to assess the performance of the prepare_proposal method in sequencer's `App`. ## Background Previous perf work has indicated this is a bottleneck. However, making that determination was done via spamoor which is slightly convoluted to run. Before working to improve the performance, we want to have a faster feedback loop on the effects of updates, hence the need for a benchmark which is easy to run and which isolates the slow function. ## Changes - Added benchmark to `app` module. Currently this has only one case: a mempool filled with transactions containing exclusively transfers. This matches the shape of the data being sent when using spamoor. - Added `benchmark` feature to enable sharing some of the existing test utils. ## Testing This is a new test. Example of running `cargo bench --features=benchmark -qp astria-sequencer app` on my Ryzen 7900X: ``` Timer precision: 10 ns astria_sequencer fastest │ slowest │ median │ mean │ samples │ iters ╰─ app │ │ │ │ │ ╰─ benchmarks │ │ │ │ │ ╰─ execute_transactions_prepare_proposal 11.63 s │ 14.68 s │ 12.74 s │ 12.88 s │ 10 │ 10 ``` Since rebasing after astriaorg#1317 has merged, the same run shows (as expected) a slowdown: ``` astria_sequencer fastest │ slowest │ median │ mean │ samples │ iters ╰─ app │ │ │ │ │ ╰─ benchmarks │ │ │ │ │ ╰─ execute_transactions_prepare_proposal 14.49 s │ 17 s │ 16.52 s │ 15.98 s │ 8 │ 8 ``` ## Related Issues Closes astriaorg#1314.
Fraser999
added a commit
to Fraser999/astria
that referenced
this pull request
Aug 25, 2024
…org#1337) ## Summary Addition of a new benchmark target to assess the performance of the prepare_proposal method in sequencer's `App`. ## Background Previous perf work has indicated this is a bottleneck. However, making that determination was done via spamoor which is slightly convoluted to run. Before working to improve the performance, we want to have a faster feedback loop on the effects of updates, hence the need for a benchmark which is easy to run and which isolates the slow function. ## Changes - Added benchmark to `app` module. Currently this has only one case: a mempool filled with transactions containing exclusively transfers. This matches the shape of the data being sent when using spamoor. - Added `benchmark` feature to enable sharing some of the existing test utils. ## Testing This is a new test. Example of running `cargo bench --features=benchmark -qp astria-sequencer app` on my Ryzen 7900X: ``` Timer precision: 10 ns astria_sequencer fastest │ slowest │ median │ mean │ samples │ iters ╰─ app │ │ │ │ │ ╰─ benchmarks │ │ │ │ │ ╰─ execute_transactions_prepare_proposal 11.63 s │ 14.68 s │ 12.74 s │ 12.88 s │ 10 │ 10 ``` Since rebasing after astriaorg#1317 has merged, the same run shows (as expected) a slowdown: ``` astria_sequencer fastest │ slowest │ median │ mean │ samples │ iters ╰─ app │ │ │ │ │ ╰─ benchmarks │ │ │ │ │ ╰─ execute_transactions_prepare_proposal 14.49 s │ 17 s │ 16.52 s │ 15.98 s │ 8 │ 8 ``` ## Related Issues Closes astriaorg#1314.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addition of a new benchmark target to assess the performance of the prepare_proposal method in sequencer's
App.Background
Previous perf work has indicated this is a bottleneck. However, making that determination was done via spamoor which is slightly convoluted to run. Before working to improve the performance, we want to have a faster feedback loop on the effects of updates, hence the need for a benchmark which is easy to run and which isolates the slow function.
Changes
appmodule. Currently this has only one case: a mempool filled with transactions containing exclusively transfers. This matches the shape of the data being sent when using spamoor.benchmarkfeature to enable sharing some of the existing test utils.Testing
This is a new test.
Example of running
cargo bench --features=benchmark -qp astria-sequencer appon my Ryzen 7900X:Since rebasing after #1317 has merged, the same run shows (as expected) a slowdown:
Related Issues
Closes #1314.