Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add execution time information to long-running test suites. #2736

Merged
merged 4 commits into from
Jun 24, 2021

Conversation

jonathanknowles
Copy link
Contributor

@jonathanknowles jonathanknowles commented Jun 24, 2021

Issue Number

ADP-978
ADP-1014

Overview

This PR adds execution timing information to long-running test suites.

It adds three kinds of information to test output:

  1. The CPU time required to execute each individual test (in milliseconds):
    prop_makeChangeForUserSpecifiedAsset_sum (189ms)
        +++ OK, passed 1000 tests.
    prop_makeChangeForUserSpecifiedAsset_length (179ms)
        +++ OK, passed 1000 tests.
  2. The total CPU time required to execute the entire test suite:
    Finished in 5.8572 seconds, used 18.3714 seconds of CPU time
    33 examples, 0 failures
  3. A list of the 10 slowest test items, when measured over the entire test suite run:
    Slow spec items:
      test/unit/Cardano/Wallet/Primitive/CoinSelection/MA/RoundRobinSpec.hs:346:9: /Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobin/Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobinSpec/Making change for many non-user-specified assets/prop_makeChangeForNonUserSpecifiedAssets_sum/ (5076ms)
      test/unit/Cardano/Wallet/Primitive/CoinSelection/MA/RoundRobinSpec.hs:344:9: /Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobin/Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobinSpec/Making change for many non-user-specified assets/prop_makeChangeForNonUserSpecifiedAssets_order/ (4903ms)
      test/unit/Cardano/Wallet/Primitive/CoinSelection/MA/RoundRobinSpec.hs:304:9: /Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobin/Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobinSpec/Making change/prop_makeChange/ (3553ms)
      test/unit/Cardano/Wallet/Primitive/CoinSelection/MA/RoundRobinSpec.hs:342:9: /Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobin/Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobinSpec/Making change for many non-user-specified assets/prop_makeChangeForNonUserSpecifiedAssets_length/ (1011ms)
      test/unit/Cardano/Wallet/Primitive/CoinSelection/MA/RoundRobinSpec.hs:300:9: /Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobin/Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobinSpec/Making change/prop_makeChange_identity/ (779ms)
      test/unit/Cardano/Wallet/Primitive/CoinSelection/MA/RoundRobinSpec.hs:302:9: /Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobin/Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobinSpec/Making change/prop_makeChange_length/ (398ms)
      test/unit/Cardano/Wallet/Primitive/CoinSelection/MA/RoundRobinSpec.hs:353:9: /Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobin/Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobinSpec/Making change for user-specified assets/prop_makeChangeForUserSpecifiedAsset_sum/ (189ms)
      test/unit/Cardano/Wallet/Primitive/CoinSelection/MA/RoundRobinSpec.hs:355:9: /Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobin/Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobinSpec/Making change for user-specified assets/prop_makeChangeForUserSpecifiedAsset_length/ (179ms)
      test/unit/Cardano/Wallet/Primitive/CoinSelection/MA/RoundRobinSpec.hs:322:9: /Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobin/Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobinSpec/Making change for coins/prop_makeChangeForCoin_sum/ (106ms)
      test/unit/Cardano/Wallet/Primitive/CoinSelection/MA/RoundRobinSpec.hs:324:9: /Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobin/Cardano.Wallet.Primitive.CoinSelection.MA.RoundRobinSpec/Making change for coins/prop_makeChangeForCoin_length/ (95ms)

@jonathanknowles jonathanknowles self-assigned this Jun 24, 2021
@jonathanknowles jonathanknowles force-pushed the jonathanknowles/test-execution-time branch from 058bfc5 to 22c5356 Compare June 24, 2021 10:37
Copy link
Contributor

@piotr-iohk piotr-iohk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jonathanknowles
Copy link
Contributor Author

bors r+

Copy link
Contributor

@paweljakubas paweljakubas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice!

@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jun 24, 2021

Build succeeded:

@iohk-bors iohk-bors bot merged commit bd95b3a into master Jun 24, 2021
@iohk-bors iohk-bors bot deleted the jonathanknowles/test-execution-time branch June 24, 2021 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants