.disableConcurrency flag not respected in quarantine mode #8087
Description
What is your Scenario?
I have a fixture with .disableConcurrency set and am running tests with -c 3 with quarantine mode on. I expect each test to run sequentially even when the test fails (to ensure each test cleans up before another starts, the actual order in which they run doesn't matter).
What is the Current behavior?
Quarantine mode ignores .disableConcurrency
What is the Expected behavior?
Quarantine mode should still respect .disableConcurrency and only one test at a time should execute.
What is the public URL of the test page? (attach your complete example)
Website not needed to demonstrate issue.
What is your TestCafe test code?
let someValue = false;
fixture("quarantineTest")
.beforeEach(async (t) => {
await t.wait(1);
someValue = false;
})
.afterEach(async (t) => {
await t.wait(1);
someValue = true;
}).disableConcurrency;
test("test1", async (t) => {
await t.wait(1000);
await t.expect(someValue).notOk("Verify value reset");
throw new Error();
});
test("test2", async (t) => {
await t.wait(1000);
await t.expect(someValue).notOk("Verify value reset");
throw new Error();
});
test("test3", async (t) => {
await t.wait(1000);
await t.expect(someValue).notOk("Verify value reset");
});
test("test4", async (t) => {
await t.wait(1000);
await t.expect(someValue).notOk("Verify value reset");
});
test("test5", async (t) => {
await t.wait(1000);
await t.expect(someValue).notOk("Verify value reset");
});
test("test6", async (t) => {
await t.wait(1000);
await t.expect(someValue).notOk("Verify value reset");
});
test("test7", async (t) => {
await t.wait(1000);
await t.expect(someValue).notOk("Verify value reset");
});
test("test8", async (t) => {
await t.wait(1000);
await t.expect(someValue).notOk("Verify value reset");
});
Your complete configuration file
Not needed
Your complete test report
See screenshots
Screenshots
Screenshot when c = 3
Screenshot when c = 1
Steps to Reproduce
Run fixture with cli:
node testcafe 'chrome:headless' 'pathToFile\quarantineTest.ts' '-c' '3' '-q' 'successThreshold=1,attemptLimit=3' '--fixture' 'quarantineTest'
Notice how tests 3-8 can be unstable
Run again with -c =1
node testcafe 'chrome:headless' 'pathToFile\quarantineTest.ts' '-c' '1' '-q' 'successThreshold=1,attemptLimit=3' '--fixture' 'quarantineTest'
Notice how there are no unstable tests. This demonstreates that quarantineMode ignores .disableConcurrency
TestCafe version
3.4.0
Node.js version
18.16.0
Command-line arguments
node testcafe 'chrome:headless' 'pathToFile\quarantineTest.ts' '-c' '3' '-q' 'successThreshold=1,attemptLimit=3' '--fixture' 'quarantineTest'
Browser name(s) and version(s)
Browser doesn't matter
Platform(s) and version(s)
Windows 11
Other
No response