Skip to content

refactor(benchmark): change benchmark CI to download data from logs in AWS#1133

Merged
glevco merged 1 commit intomasterfrom
refactor/bench-data
Oct 3, 2024
Merged

refactor(benchmark): change benchmark CI to download data from logs in AWS#1133
glevco merged 1 commit intomasterfrom
refactor/bench-data

Conversation

@glevco
Copy link
Contributor

@glevco glevco commented Sep 9, 2024

Motivation

Currently, CI benchmark directly connects to testnet to download benchmark data (it performs a sync all the way up to 20k blocks). This PR changes it to use the latest load-from-logs CLI tool update, making the CI run faster and more reliably. Then, we'll also be able to introduce new CI benchmarks in other PRs, including for example syncing a "dummy" network with more transactions than blocks.

Acceptance Criteria

  • Change Sync-v2 benchmark CI to download benchmark data from a logs file in AWS instead of directly connecting to the network.
  • Update config files accordingly and improve organization.

Checklist

  • If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

@glevco glevco self-assigned this Sep 9, 2024
@github-actions
Copy link

github-actions bot commented Sep 9, 2024

🐰 Bencher Report

Branchrefactor/bench-data
Testbedubuntu-22.04
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)
Lower Boundary
nanoseconds (ns)
(Limit %)
Upper Boundary
nanoseconds (ns)
(Limit %)
sync-v2 (up to 20000 blocks)📈 view plot
🚷 view threshold
101,471,694,713.76
(-0.55%)
91,826,613,182.35
(90.49%)
112,232,527,222.88
(90.41%)
🐰 View full continuous benchmarking report in Bencher

@codecov
Copy link

codecov bot commented Sep 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.36%. Comparing base (7f53032) to head (63d11c1).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1133      +/-   ##
==========================================
+ Coverage   84.33%   84.36%   +0.02%     
==========================================
  Files         317      317              
  Lines       24297    24297              
  Branches     3697     3697              
==========================================
+ Hits        20491    20498       +7     
+ Misses       3083     3078       -5     
+ Partials      723      721       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@glevco glevco marked this pull request as ready for review September 9, 2024 18:32
Copy link
Member

@jansegre jansegre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call it something else other than "sync_v2" now? Using load-from-logs shouldn't trigger any sync code, right? Maybe "load_sample". Other than that it looks good to me.

@glevco
Copy link
Contributor Author

glevco commented Sep 18, 2024

@jansegre

Should we call it something else other than "sync_v2" now? Using load-from-logs shouldn't trigger any sync code, right? Maybe "load_sample". Other than that it looks good to me.

The benchmark is still the same, it uses Sync-v2. I only changed the way the server node boots up (from syncing with the network vs from logs). Then, a second node connects to the server node and the measurement is made on the sync of that second node.

@glevco glevco force-pushed the refactor/bench-data branch from c54fde5 to 63d11c1 Compare October 3, 2024 21:10
@glevco glevco merged commit 115569a into master Oct 3, 2024
@glevco glevco deleted the refactor/bench-data branch October 3, 2024 22:25
@jansegre jansegre mentioned this pull request Oct 4, 2024
2 tasks
@jansegre jansegre mentioned this pull request Dec 11, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants