Improve clarity of test failures within prop_performSelection.
#2960
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jonathanknowles
changed the title
prop_performSelection.prop_performSelection.
jonathanknowles
force-pushed
the
jonathanknowles/adp-1037/test-coin-selection-3
branch
from
60b5c41 to
c1cddc5
Compare
It's not actually necessary to blind this argument, since it's already blinded by the outer properties that call this property, and those properties fully determine the value of the argument.
We now can easily verify that each of these properties is monadic. Later commits will convert each of these properties to a non-monadic style, enabling us to use the `report` and `verify` combinators to clarify the output of failures.
jonathanknowles
force-pushed
the
jonathanknowles/adp-1037/test-coin-selection-3
branch
from
bd73303 to
96b6218
Compare
jonathanknowles
force-pushed
the
jonathanknowles/adp-1037/test-coin-selection-3
branch
from
96b6218 to
cb2ce80
Compare
jonathanknowles
force-pushed
the
jonathanknowles/adp-1037/test-coin-selection-3
branch
from
cb2ce80 to
9409c10
Compare
We can't re-use the `report` name here, as this is already imported. So we use `transformationReport` instead.
jonathanknowles
force-pushed
the
jonathanknowles/adp-1037/test-coin-selection-3
branch
from
9409c10 to
6c639b8
Compare
sevanspowell
approved these changes
Oct 8, 2021
|
bors r+ |
iohk-bors bot
added a commit
that referenced
this pull request
Oct 8, 2021
2960: Improve clarity of test failures within `prop_performSelection`. r=jonathanknowles a=jonathanknowles ## Issue Number ADP-1037 ## Summary This PR uses the [**`report`**](https://github.com/input-output-hk/cardano-wallet/blob/b1ded0657c3d7fafcd7bbac15d9cd07478e34058/lib/test-utils/src/Test/QuickCheck/Extra.hs#L378) and [**`verify`**](https://github.com/input-output-hk/cardano-wallet/blob/b1ded0657c3d7fafcd7bbac15d9cd07478e34058/lib/test-utils/src/Test/QuickCheck/Extra.hs#L386) combinators to clarify the output of failures for the [`BalanceSpec.prop_performSelection`](https://github.com/input-output-hk/cardano-wallet/blob/942f12ae838fbce0ba00ce0109c7e26ba53e4318/lib/core/test/unit/Cardano/Wallet/Primitive/CoinSelection/BalanceSpec.hs#L821) property. Currently, failures of this property are hard to debug, since even if only one assertion fails, then the names of all successful assertions prior to the failed assertion will also be appended to the output. (This is a limitation of `assertWith`.) ## Details * The [**`report`**](https://github.com/input-output-hk/cardano-wallet/blob/b1ded0657c3d7fafcd7bbac15d9cd07478e34058/lib/test-utils/src/Test/QuickCheck/Extra.hs#L378) combinator automatically pretty-prints values that are passed to it, making it much easier to inspect the internal structure of counterexample values. * The [**`verify`**](https://github.com/input-output-hk/cardano-wallet/blob/b1ded0657c3d7fafcd7bbac15d9cd07478e34058/lib/test-utils/src/Test/QuickCheck/Extra.hs#L386) combinator produces clearer output than `assertWith`, since `verify` produces output that's **_specific_** to the condition that fails, whereas `assertWith` can only append to the current counterexample text, making it unsuitable for use in situations where there are many things to verify. Both of these combinators require a non-monadic `Property` context, so this PR also converts most of `prop_performSelection` to a non-monadic style. (Only a small portion of this property actually needs to be run within `monadicIO`.) ## Sample output  Co-authored-by: Jonathan Knowles <[email protected]>
|
Build failed: There were no actual test failures in this merge run. This looks like a spurious failure caused by file locks: |
|
bors r+ |
|
Build succeeded: |
iohk-bors
bot
deleted the
jonathanknowles/adp-1037/test-coin-selection-3
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue Number
ADP-1037
Summary
This PR uses the
reportandverifycombinators to clarify the output of failures for theBalanceSpec.prop_performSelectionproperty.Currently, failures of this property are hard to debug, since even if only one assertion fails, then the names of all successful assertions prior to the failed assertion will also be appended to the output. (This is a limitation of
assertWith.)Details
reportcombinator automatically pretty-prints values that are passed to it, making it much easier to inspect the internal structure of counterexample values.verifycombinator produces clearer output thanassertWith, sinceverifyproduces output that's specific to the condition that fails, whereasassertWithcan only append to the current counterexample text, making it unsuitable for use in situations where there are many things to verify.Both of these combinators require a non-monadic
Propertycontext, so this PR also converts most ofprop_performSelectionto a non-monadic style. (Only a small portion of this property actually needs to be run withinmonadicIO.)Sample output