Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up RequestLib tests #162

Merged
merged 8 commits into from
Dec 21, 2023
Merged

Speed up RequestLib tests #162

merged 8 commits into from
Dec 21, 2023

Conversation

StuAA78
Copy link
Contributor

@StuAA78 StuAA78 commented Mar 13, 2023

DEFRA/water-abstraction-team#98

For those paying attention when running our unit tests, they appear to stall when it comes to test/lib/request.lib.test.js. This is because we have tests that ensure the module behaves as expected when requests to other APIs time out or fail.

One of the reasons we love Got is because retries are built-in and easy to configure. But that means we have to allow Got to retry failures when running the unit tests.

The reason for the stalling is because the time between retries is based on a 'computed value';

((2 ** (attemptCount - 1)) * 1000) + noise

This means the delay increases exponentially. That's great for what we do in the app. A 'blip' when talking to an external API is smoothed over and all the user sees is something takes a few seconds longer than normal. But in our unit tests it causes some to appear to stall.

Thankfully, the brilliant @StuAA78 spotted that we can override the computed value by setting the backoffLimit.

The upper limit of the computedValue.

If we set this when running the unit tests the time between retries will be limited to 50ms.

So, this change updates RequestLib to make it easy to access the default options it uses so we can then apply backofflimit to the retry options.


Before changes

  • Test duration: 40059 ms
  • Test duration: 40057 ms
  • Test duration: 40116 ms

Average 40077 ms (40 secs)

After changes

  • Test duration: 16960 ms
  • Test duration: 16948 ms
  • Test duration: 17009 ms

Average 16972 ms (16 secs)

@StuAA78 StuAA78 added the housekeeping Refactoring, tidying up or other work which supports the project label Mar 13, 2023
@StuAA78 StuAA78 self-assigned this Mar 13, 2023
@Cruikshanks Cruikshanks assigned Cruikshanks and unassigned StuAA78 Sep 9, 2023
@Cruikshanks Cruikshanks force-pushed the speed-up-request-lib-tests branch from 02b6f78 to 8ba86ff Compare December 19, 2023 17:04
@Cruikshanks Cruikshanks marked this pull request as ready for review December 19, 2023 17:53
@Cruikshanks
Copy link
Member

If it helps, comparing the unit test times on this branch compared to others it looks like this change will cut 20ish secs of build time 🤷🎉

Copy link
Contributor

@Jozzey Jozzey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Cruikshanks Cruikshanks merged commit 8506950 into main Dec 21, 2023
6 checks passed
@Cruikshanks Cruikshanks deleted the speed-up-request-lib-tests branch December 21, 2023 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
housekeeping Refactoring, tidying up or other work which supports the project
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants