Skip to content

perf(sequencer): add benchmark for prepare_proposal (ENG-660)#1329

Closed
Fraser999 wants to merge 1 commit intoastriaorg:mainfrom
Fraser999:fraser/add-sequencer-benchmark
Closed

perf(sequencer): add benchmark for prepare_proposal (ENG-660)#1329
Fraser999 wants to merge 1 commit intoastriaorg:mainfrom
Fraser999:fraser/add-sequencer-benchmark

Conversation

@Fraser999
Copy link
Contributor

@Fraser999 Fraser999 commented Aug 1, 2024

Summary

Addition of a new benchmark target to assess the performance of the prepare_proposal method in sequencer's App.

Background

Previous perf work has indicated this is a bottleneck. However, making that determination was done via spamoor which is slightly convoluted to run. Before working to improve the performance, we want to have a faster feedback loop on the effects of updates, hence the need for a benchmark which is easy to run and which isolates the slow function.

Changes

  • Added benchmark to app module. Currently this has only one case: a mempool filled with transactions containing exclusively transfers. This matches the shape of the data being sent when using spamoor.
  • Added benchmark feature to enable sharing some of the existing test utils.

Testing

This is a new test.

Example of running cargo bench --features=benchmark -qp astria-sequencer app on my Ryzen 7900X:

Timer precision: 10 ns
astria_sequencer                                fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ app                                                        │               │               │               │         │
   ╰─ benchmarks                                              │               │               │               │         │
      ╰─ execute_transactions_prepare_proposal  11.63 s       │ 14.68 s       │ 12.74 s       │ 12.88 s       │ 10      │ 10

Since rebasing after #1317 has merged, the same run shows (as expected) a slowdown:

astria_sequencer                                fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ app                                                        │               │               │               │         │
   ╰─ benchmarks                                              │               │               │               │         │
      ╰─ execute_transactions_prepare_proposal  14.49 s       │ 17 s          │ 16.52 s       │ 15.98 s       │ 8       │ 8

Related Issues

Closes #1314.

@Fraser999 Fraser999 requested a review from a team as a code owner August 1, 2024 10:06
@Fraser999 Fraser999 requested a review from SuperFluffy August 1, 2024 10:06
@github-actions github-actions bot added the sequencer pertaining to the astria-sequencer crate label Aug 1, 2024
@Fraser999 Fraser999 force-pushed the fraser/add-sequencer-benchmark branch from 28ff254 to 9b039da Compare August 1, 2024 20:47
@Fraser999
Copy link
Contributor Author

Closed in favour of #1337 (this PR was cut from the wrong repo).

@Fraser999 Fraser999 closed this Aug 1, 2024
@Fraser999 Fraser999 deleted the fraser/add-sequencer-benchmark branch August 1, 2024 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sequencer pertaining to the astria-sequencer crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provide faster feedback loop for assessing performance of sequencer

1 participant