Improve speed of cucumber test suite #6362
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mpw5
force-pushed
the
mw/faster-cucumbers
branch
6 times, most recently
from
7dba4d5 to
c3b6e2c
Compare
`Capybara.default_max_wait_time` is intended as a threshold for Capybara to use when waiting for an Ajax request to complete: https://github.com/teamcapybara/capybara/blob/master/README.md#asynchronous-javascript-ajax-and-friends It is currently configured as 10 seconds in CCCD, but when used correctly will return as soon as the request is complete/expected element is displayed (ie it acts as a maximum time before erroring). In one step definition, `default_max_wait_time` is being used to set a the duration of a `sleep`. This means that all tests that use that step will pause for 10 seconds, even though it is likely that the page will be ready long before then. In an attempt to speed up the test suite, this commit sets that sleep to 1 second, which does not seem to cause any failures.
Replaces a number of `sleep` calls in feature test step definitions with `using_wait_time` blocks. `using_wait_time` returns early if the assertion is successful, so using this approach should be quicker than sleeping for an absolute amount of time. In a couple of places where a `using_wait_time` block isn't suitable, the `sleep` statement has been removed as an experiment to see if there is any impact.
Brings the time to run the feature test suite down from ~12.5 minutes to ~10 minutes.
mpw5
force-pushed
the
mw/faster-cucumbers
branch
6 times, most recently
from
d26f1b2 to
e1e4a46
Compare
When the cucumber test suite starts, a large volume of production seed data is loaded into the database. This includes a full list of offences for nine fee schemes, however only ten of those offences are used in tests. In an attempt to speed up the running of the test suite, this amends `db/seeds/scheme_11.rb` to load a bespoke test-only version of the offence data for AGFS fee scheme 11, containing only those ten offences. This set of data is copied for subsequent AGFS fee schemes so the total number of records being created is reduced from ~6,500 to 50. * We could consider doing something similar for other seed data. * We could consider not seeding the database at all and use factories to generate required data instead.
mpw5
force-pushed
the
mw/faster-cucumbers
branch
from
fce47a1 to
e275ce9
Compare
|
mpw5
changed the title
Obsiye
approved these changes
Jan 3, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
A speculative attempt to speed up the CCCD feature test suite
Why
The test suite is regularly taking 13+ minutes to run on CircleCI, and has taken over 15 minutes. It would be nice if it was quicker...
How
Various attempts, including:
sleepstatements in feature tests (this appears to casuse only a small improvement)increase CircleCI resource class size(this appear to have no affect on runtime)Outcome
Rough timings:
8x parellel + same seed data = ~10 minutes
6x parellel + reduced seed data = ~8 minutes
8x parellel + reduced seed data = ~7 minutes