Skip to content

Foundry test suite clean up#3830

Closed
maurelian wants to merge 13 commits intodevelopfrom
test/refactor-1
Closed

Foundry test suite clean up#3830
maurelian wants to merge 13 commits intodevelopfrom
test/refactor-1

Conversation

@maurelian
Copy link
Contributor

@maurelian maurelian commented Nov 1, 2022

Description

Furthers the work of reorganizing the forge tests, which was begun in #3611.

Test organizing principles

  1. Uses contracts to organize the test suite similar to how mocha uses describe.
  2. Every non-trivial state changing function should have a contract for happy and sad path tests. This helps to make it very obvious where there are not sad tests (I haven't added any new tests, so there are quite a few empty placeholder contracts).
  3. Simpler functions like getters and setters are grouped together into test contracts.
  4. All contracts and functions should be consistently named following a convention.

Test naming convention

Test function names are split by underscores, into 3 or 4 parts. An example function name is test_onlyOwner_callerIsNotOwner_reverts().

The parts are: [method]_[FunctionName]_[reason]_[success], where:

  1. [method] is either test or testFuzz
  2. [FunctionName] is the name of the function or higher level behavior being tested.
  3. [reason] is an optional description for the behavior being tested.
  4. [success] must be one of success or reverts or _fails.

Name validation

I also wrote a quick and dirty script which can be run to validate function names and identify ones which are not yet compliant. From within the bedrock package:
ts-node scripts/forges-test-names.ts.

It will error, because there are still test files which haven't yet been cleaned up.

@changeset-bot
Copy link

changeset-bot bot commented Nov 1, 2022

⚠️ No Changeset found

Latest commit: 55306da

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions github-actions bot added 2-reviewers A-pkg-contracts-bedrock Area: packages/contracts-bedrock labels Nov 1, 2022
@mergify
Copy link
Contributor

mergify bot commented Nov 1, 2022

This PR changes implementation code, but doesn't include a changeset. Did you forget to add one?

@maurelian maurelian marked this pull request as ready for review November 1, 2022 20:10
@maurelian maurelian changed the title Test Clean Up Foundry test suite clean up Nov 1, 2022
@maurelian maurelian requested review from tynes and removed request for mslipper November 1, 2022 20:19
@mergify
Copy link
Contributor

mergify bot commented Nov 2, 2022

Hey @maurelian! This PR has merge conflicts. Please fix them before continuing review.

@mergify mergify bot added the conflict label Nov 2, 2022
Copy link
Contributor

@smartcontracts smartcontracts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like this refactor. It makes holes in test coverage much more obvious. Thanks!

}
}

contract Encoding_EncodeVersionedNonce_TestFail is Encoding_TestInit {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this included if there are none?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intention is to express that I've actually thought about whether or not there are any sad path tests that could be written here, and determined that there are none.
I like having it, but I'm not attached to it.

@@ -0,0 +1,77 @@
import fs from 'fs'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this script run as part of CI?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eventually yes, but I haven't yet gone through all the test files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would fail if run in CI now.

@tynes
Copy link
Contributor

tynes commented Nov 2, 2022

Generally makes sense, I think we need to document the guidelines in the readme that you posted here

Also this PR needs a rebase to fix the conflicts with the more files

@maurelian maurelian force-pushed the test/refactor-1 branch 2 times, most recently from 4ec2ee1 to a61feaa Compare November 3, 2022 01:36
@maurelian maurelian requested a review from tynes November 3, 2022 01:37
@mergify mergify bot removed the conflict label Nov 3, 2022
@maurelian
Copy link
Contributor Author

@tynes
I documented the testing conventions here: a61feaa (#3830)

@mergify
Copy link
Contributor

mergify bot commented Nov 3, 2022

Hey @maurelian! This PR has merge conflicts. Please fix them before continuing review.

@mergify mergify bot added the conflict label Nov 3, 2022
@mergify mergify bot removed the conflict label Nov 3, 2022
@maurelian maurelian mentioned this pull request Nov 3, 2022
14 tasks
@mergify
Copy link
Contributor

mergify bot commented Nov 3, 2022

Hey @maurelian! This PR has merge conflicts. Please fix them before continuing review.

@mergify mergify bot added the conflict label Nov 3, 2022
@maurelian
Copy link
Contributor Author

maurelian commented Nov 8, 2022

Not a priority right now. Setting to draft.

@maurelian maurelian marked this pull request as draft November 9, 2022 01:06
@github-actions
Copy link
Contributor

This PR is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days.

@github-actions github-actions bot added the Stale label Nov 23, 2022
@maurelian maurelian closed this Nov 23, 2022
@mergify mergify bot removed the conflict label Nov 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-pkg-contracts-bedrock Area: packages/contracts-bedrock

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants