Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add latency benchmark for shelley #1845

Merged
merged 6 commits into from
Jul 14, 2020
Merged

Conversation

rvl
Copy link
Contributor

@rvl rvl commented Jul 2, 2020

Relates to #1825.

Overview

  • Ports latency benchmarks to shelley backend.
  • Refactors duplicate code into a shared module.
  • Adds listStakePools to benchmarks

@rvl rvl self-assigned this Jul 2, 2020
@rvl rvl force-pushed the rvl/1825/shelley-bench-latency branch from f347c0e to 50fdd86 Compare July 2, 2020 07:09
@rvl rvl force-pushed the rvl/1825/estimate-fee branch from f0f8e6a to 414ba03 Compare July 2, 2020 07:10
@rvl rvl added the IMPROVEMENT Mark a PR as an improvement, for auto-generated CHANGELOG label Jul 2, 2020
@rvl rvl force-pushed the rvl/1825/shelley-bench-latency branch from 50fdd86 to c39123c Compare July 2, 2020 07:11
@rvl rvl marked this pull request as ready for review July 2, 2020 07:12
@rvl rvl force-pushed the rvl/1825/shelley-bench-latency branch 2 times, most recently from dc31f7b to d185ab5 Compare July 2, 2020 08:44
@rvl rvl requested review from paweljakubas and piotr-iohk July 3, 2020 12:27
@piotr-iohk
Copy link
Contributor

Probably not related to this pr (?), but I got this when running shelley latency benchmark locally:

+++ Run benchmark - shelley
    Non-cached run
        getNetworkInfo      - 0.1 ms
    Latencies for 2 fixture wallets scenario
        listWallets         - 8.2 ms
        getWallet           - 3.9 ms
        getUTxOsStatistics  - 1.4 ms
        listAddresses       - 4.9 ms
        listTransactions    - 2.3 ms
        postTransactionFee  - 196.7 ms
        getNetworkInfo      - 0.0 ms
    Latencies for 10 fixture wallets scenario
SQLite3 returned ErrorMisuse while attAction failed. Here are the captured logs:
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":65,"logMsg":"Received API request"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":65,"logMsg":"[GET] /v2/network/information"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":65,"logMsg":"200 OK in 0.000069772s"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":65,"logMsg":"Completed response to API request"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":66,"logMsg":"Received API request"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":66,"logMsg":"[GET] /v2/network/information"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":66,"logMsg":"200 OK in 0.000042714s"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":66,"logMsg":"Completed response to API request"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":67,"logMsg":"Received API request"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":67,"logMsg":"[GET] /v2/network/information"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":67,"logMsg":"200 OK in 0.000042468s"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":67,"logMsg":"Completed response to API request"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":68,"logMsg":"Received API request"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":68,"logMsg":"[GET] /v2/network/information"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":68,"logMsg":"200 OK in 0.000041473s"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":68,"logMsg":"Completed response to API request"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":69,"logMsg":"Received API request"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":69,"logMsg":"[GET] /v2/network/information"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":69,"logMsg":"200 OK in 0.000040221s"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":69,"logMsg":"Completed response to API request"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":70,"logMsg":"Received API request"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":70,"logMsg":"[GET] /v2/network/information"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":70,"logMsg":"200 OK in 0.000039737s"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":70,"logMsg":"Completed response to API request"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":71,"logMsg":"Received API request"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":71,"logMsg":"[GET] /v2/network/information"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":71,"logMsg":"200 OK in 0.000040118s"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":71,"logMsg":"Completed response to API request"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":72,"logMsg":"Received API request"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":72,"logMsg":"[GET] /v2/network/information"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":72,"logMsg":"200 OK in 0.000039731s"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":72,"logMsg":"Completed response to API request"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":73,"logMsg":"Received API request"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":73,"logMsg":"[GET] /v2/network/information"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":73,"logMsg":"200 OK in 0.000042853s"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":73,"logMsg":"Completed response to API request"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":74,"logMsg":"Received API request"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":74,"logMsg":"[GET] /v2/network/information"}
[:Info:75] [2020-07-03 13:08:49.97 UTC] {"requestId":74,"logMsg":"200 OK in 0.000039623s"}
[:Debug:75] [2020-07-03 13:08:49.97 UTC] {"requestId":74,"logMsg":"Completed response to API request"}
[:Debug:1247] [2020-07-03 13:08:50.01 UTC] {"requestId":0,"logMsg":"Received API request"}
[:Info:1247] [2020-07-03 13:08:50.01 UTC] {"requestId":0,"logMsg":"[POST] /v2/wallets"}
latency: DecodeFailure "Something went wrong"

@rvl rvl marked this pull request as draft July 3, 2020 14:37
@rvl rvl marked this pull request as ready for review July 3, 2020 14:40
Copy link
Contributor

@paweljakubas paweljakubas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rvl
Copy link
Contributor Author

rvl commented Jul 4, 2020

Yes thanks @paweljakubas I should do that. (edit: it's done)

Base automatically changed from rvl/1825/estimate-fee to master July 7, 2020 14:53
@rvl rvl force-pushed the rvl/1825/shelley-bench-latency branch from 3e2f951 to 87fa6b3 Compare July 8, 2020 00:54
@rvl
Copy link
Contributor Author

rvl commented Jul 8, 2020

bors r+

iohk-bors bot added a commit that referenced this pull request Jul 8, 2020
1845: Add latency benchmark for shelley r=rvl a=rvl

Relates to #1825.

### Overview

- Ports latency benchmarks to shelley backend.
- Refactors duplicate code into a shared module.
- Adds listStakePools to benchmarks

Co-authored-by: Rodney Lorrimar <[email protected]>
Co-authored-by: Piotr Stachyra <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jul 8, 2020

Build failed

@rvl rvl force-pushed the rvl/1825/shelley-bench-latency branch from 87fa6b3 to dd15e8e Compare July 13, 2020 10:31
@rvl
Copy link
Contributor Author

rvl commented Jul 13, 2020

bors r+

iohk-bors bot added a commit that referenced this pull request Jul 13, 2020
1845: Add latency benchmark for shelley r=rvl a=rvl

Relates to #1825.

### Overview

- Ports latency benchmarks to shelley backend.
- Refactors duplicate code into a shared module.
- Adds listStakePools to benchmarks

1899: Enable `STAKE_POOLS_LIST` tests to run in isolation. r=jonathanknowles a=jonathanknowles

# Issue Number

#1819 

# Overview

The very first test in the `STAKE_POOLS_LIST_01` series has implicit dependency on time. If run _too soon_ (as is the case if the test is run in isolation), such that the wallet has not yet had a chance to catch up with tip, then the wallet API will return an empty list, which will cause a test failure.

To fix this, this PR nests the test inside an `eventually` combinator, which allows the test to be run in isolation (just like the others in the `STAKE_POOLS_LIST_01` series).

Additional changes:

- [x] Fixes some spelling errors.
- [x] Reformatting to comply with our coding standards.


Co-authored-by: Rodney Lorrimar <[email protected]>
Co-authored-by: Piotr Stachyra <[email protected]>
Co-authored-by: Jonathan Knowles <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jul 13, 2020

Build failed (retrying...)

iohk-bors bot added a commit that referenced this pull request Jul 13, 2020
1845: Add latency benchmark for shelley r=rvl a=rvl

Relates to #1825.

### Overview

- Ports latency benchmarks to shelley backend.
- Refactors duplicate code into a shared module.
- Adds listStakePools to benchmarks

Co-authored-by: Rodney Lorrimar <[email protected]>
Co-authored-by: Piotr Stachyra <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jul 13, 2020

Build failed

@rvl
Copy link
Contributor Author

rvl commented Jul 14, 2020

bors r+

iohk-bors bot added a commit that referenced this pull request Jul 14, 2020
1845: Add latency benchmark for shelley r=rvl a=rvl

Relates to #1825.

### Overview

- Ports latency benchmarks to shelley backend.
- Refactors duplicate code into a shared module.
- Adds listStakePools to benchmarks

Co-authored-by: Rodney Lorrimar <[email protected]>
Co-authored-by: Piotr Stachyra <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jul 14, 2020

Build failed

@rvl
Copy link
Contributor Author

rvl commented Jul 14, 2020

bors r+

@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jul 14, 2020

@iohk-bors iohk-bors bot merged commit 52966c3 into master Jul 14, 2020
@iohk-bors iohk-bors bot deleted the rvl/1825/shelley-bench-latency branch July 14, 2020 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IMPROVEMENT Mark a PR as an improvement, for auto-generated CHANGELOG
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants