refactor(benchmark): change benchmark CI to download data from logs in AWS#1133
refactor(benchmark): change benchmark CI to download data from logs in AWS#1133
Conversation
|
| Branch | refactor/bench-data |
| Testbed | ubuntu-22.04 |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result nanoseconds (ns) (Result Δ%) | Lower Boundary nanoseconds (ns) (Limit %) | Upper Boundary nanoseconds (ns) (Limit %) |
|---|---|---|---|---|
| sync-v2 (up to 20000 blocks) | 📈 view plot 🚷 view threshold | 101,471,694,713.76 (-0.55%) | 91,826,613,182.35 (90.49%) | 112,232,527,222.88 (90.41%) |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #1133 +/- ##
==========================================
+ Coverage 84.33% 84.36% +0.02%
==========================================
Files 317 317
Lines 24297 24297
Branches 3697 3697
==========================================
+ Hits 20491 20498 +7
+ Misses 3083 3078 -5
+ Partials 723 721 -2 ☔ View full report in Codecov by Sentry. |
jansegre
left a comment
There was a problem hiding this comment.
Should we call it something else other than "sync_v2" now? Using load-from-logs shouldn't trigger any sync code, right? Maybe "load_sample". Other than that it looks good to me.
The benchmark is still the same, it uses Sync-v2. I only changed the way the server node boots up (from syncing with the network vs from logs). Then, a second node connects to the server node and the measurement is made on the sync of that second node. |
c54fde5 to
63d11c1
Compare
Motivation
Currently, CI benchmark directly connects to testnet to download benchmark data (it performs a sync all the way up to 20k blocks). This PR changes it to use the latest load-from-logs CLI tool update, making the CI run faster and more reliably. Then, we'll also be able to introduce new CI benchmarks in other PRs, including for example syncing a "dummy" network with more transactions than blocks.
Acceptance Criteria
Checklist
master, confirm this code is production-ready and can be included in future releases as soon as it gets merged