You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've searched existing issues and found nothing related to my issue.
Describe the bug
When working on my fork of the whole bruno project to replicate a defect that my colleague is having, I checked out and started running the test suite as detailed in packages/bruno-tests/readme.md, but when I tried to run the Local tests, some of them work, but some of them are failing.
This is probably something where if we fixed this, and part of that would probably entail making the Local suite also run inside the github action, it might help accelerate any/all developers working on the project since they would be able to have the suite run much faster and can continue working when their internet dies or is slow.
I've done a bit of an investigation, but the answer will probably require some changes to both the collections and the testbench, as well as a bit of an extension, e.g. have a very barebones graphql implementation instead of using spacex's in Local/Prod tests. I've detailed a bit of work required to be done below
Failing Steps
Looking through each test case(s) folder by folder in the tree structure:
auth/: ~ 75ms locally, ~ 4000 ms using Prod.
echo/: This fails out the box since it's using echo-host locally, when the prod has this. With a slight modification (which can come in a pull request), 2586 ms using Prod, ~150ms if I remove some of the xml scenarios that seem to hang locally.
graphql/: This calls spacex - 837 ms, will require a bit inside the Local to emulate graphql, but should be a lot faster.
preview/: Both making network requests, ~ 500ms
redirections/: ~ 41ms ~ 2016 ms using Prod
scripting/: 12 failed, ~ 1400ms, ~11,494 ms using Prod but nearly all passed. One with a couple of changes needed. There's a setTimeout with 1000ms which could be shaved down and the axios-pre-req-script.bru uses a hardcoded testbench too.
string interpolation/: 3 failed, 39ms vs ~1300ms using Prod, all passing.
In total with failures, getting the local experience of ~2567 ms vs ~19101 ms using Prod without major changes. Which once we swap out the timeout and network requests, it should be able to run around a second.
Below is the local run visualised using a helpful website to show which tests dominate the time taken in any test suite.
.bru file to reproduce the bug
No response
Screenshots/Live demo link
There is nothing to screenshot, this is a test suite speedup.
The text was updated successfully, but these errors were encountered:
I have checked the following:
Describe the bug
When working on my fork of the whole bruno project to replicate a defect that my colleague is having, I checked out and started running the test suite as detailed in
packages/bruno-tests/readme.md
, but when I tried to run the Local tests, some of them work, but some of them are failing.This is probably something where if we fixed this, and part of that would probably entail making the Local suite also run inside the github action, it might help accelerate any/all developers working on the project since they would be able to have the suite run much faster and can continue working when their internet dies or is slow.
I've done a bit of an investigation, but the answer will probably require some changes to both the collections and the testbench, as well as a bit of an extension, e.g. have a very barebones graphql implementation instead of using spacex's in Local/Prod tests. I've detailed a bit of work required to be done below
Failing Steps
Looking through each test case(s) folder by folder in the tree structure:
auth/
: ~ 75ms locally, ~ 4000 ms using Prod.echo/
: This fails out the box since it's using echo-host locally, when the prod has this. With a slight modification (which can come in a pull request), 2586 ms using Prod, ~150ms if I remove some of the xml scenarios that seem to hang locally.graphql/
: This calls spacex - 837 ms, will require a bit inside the Local to emulate graphql, but should be a lot faster.preview/
: Both making network requests, ~ 500msredirections/
: ~ 41ms ~ 2016 ms using Prodscripting/
: 12 failed, ~ 1400ms, ~11,494 ms using Prod but nearly all passed. One with a couple of changes needed. There's asetTimeout
with 1000ms which could be shaved down and theaxios-pre-req-script.bru
uses a hardcoded testbench too.string interpolation/
: 3 failed, 39ms vs ~1300ms using Prod, all passing.In total with failures, getting the local experience of ~2567 ms vs ~19101 ms using Prod without major changes. Which once we swap out the timeout and network requests, it should be able to run around a second.
Below is the local run visualised using a helpful website to show which tests dominate the time taken in any test suite.
.bru file to reproduce the bug
No response
Screenshots/Live demo link
There is nothing to screenshot, this is a test suite speedup.
The text was updated successfully, but these errors were encountered: