-
Notifications
You must be signed in to change notification settings - Fork 998
docs: ux fuzz invariant #880
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
0eecb54
Docs for in-line test configuration.
0xMelkor 54e6740
Added description about base config inheritance
0xMelkor 0c549ed
Rewording
0xMelkor d165214
Pages that describe "tests" now reference pages that describe how to …
0xMelkor aa6ff26
Config in block comments explained
0xMelkor File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| ## In-line test configuration | ||
| Foundry users are enabled to specify overall test configurations, using a combination of ENV variables and config statements in the `foundry.toml`. Checkout the [📚 Testing reference](./testing) for a detailed description. | ||
|
|
||
| Despite this may work in the general case, some tests may need finer control over their configuration. For such reason Forge provides a way to specify per-test configs for invariant and fuzz testing scenarios. | ||
|
|
||
| Users can in-line test config statements directly in Solidity comments. This would affect the behavior of the `forge test` command for a specific test instance, as illustrated in the example below. | ||
|
|
||
| ```solidity | ||
| contract MyTest is Test { | ||
| /// forge-config: default.fuzz.runs = 100 | ||
| /// forge-config: ci.fuzz.runs = 500 | ||
| function test_SimpleFuzzTest(uint256 x) public { | ||
| // --- snip --- | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| What we are asking here is to run our fuzzer `100` and `500` times for the `default` and `ci` profiles respectively. The interesting fact is that this would override any fuzz `runs` setup existing at a global level. All other configs would be inherited from the global context, making this acting as a fallback for all possible configurations. | ||
|
|
||
| ### Block comments | ||
| In-line test configurations can also be expressed in block comments, as illustrated in the example. | ||
|
|
||
| ```solidity | ||
| contract MyTest is Test { | ||
| /** | ||
| * forge-config: default.fuzz.runs = 1024 | ||
| * forge-config: default.fuzz.max-test-rejects = 500 | ||
| */ | ||
| function test_SimpleFuzzTest(uint256 x) public { | ||
| // --- snip --- | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### In-line fuzz configs | ||
| Users can specify the configs described in the table. Each statement must have a prefix of the form `forge-config: ${PROFILE}.fuzz.` | ||
|
|
||
| | Parameter | Type | Description | | ||
| |-|-|-| | ||
| |`runs`|integer|The amount of fuzz runs to perform for this specific test case [📚 ref](./testing#runs).| | ||
| |`max-test-rejects`|integer|The maximum number of combined inputs that may be rejected before the test as a whole aborts [📚 ref](./testing#max_test_rejects).| | ||
|
|
||
| Fuzz config example | ||
| ```solidity | ||
| contract MyFuzzTest is Test { | ||
| /// forge-config: default.fuzz.runs = 100 | ||
| /// forge-config: default.fuzz.max-test-rejects = 2 | ||
| function test_InlineConfig(uint256 x) public { | ||
| // --- snip --- | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### In-line invariant configs | ||
| Users can specify the configs described in the table. Each statement must have a prefix of the form `forge-config: ${PROFILE}.invariant.` | ||
|
|
||
| | Parameter | Type | Description | | ||
| |-|-|-| | ||
| |`runs`|integer|The amount of invariant runs to perform for this specific test case [📚 ref](./testing#runs-1). | ||
| |`depth`|integer|The number of calls executed to attempt to break invariant in one run [📚 ref](./testing#depth). | ||
| |`fail-on-revert`|boolean|Fails the invariant fuzzing if a revert occurs [📚 ref](./testing#fail_on_revert). | ||
| |`call-override`|boolean|Overrides unsafe external calls when running invariant test [📚 ref](./testing#call_override). | ||
|
|
||
| Invariant config example | ||
| ```solidity | ||
| contract MyInvariantTest is Test { | ||
| /// forge-config: default.invariant.runs = 100 | ||
| /// forge-config: default.invariant.depth = 2 | ||
| /// forge-config: default.invariant.fail-on-revert = false | ||
| /// forge-config: default.invariant.call-override = true | ||
| function invariant_InlineConfig() public { | ||
| // --- snip --- | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.