Mocha 8 parallel mode is not supported

[Xotabu4]

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Running tests using mocha 8 parallel feature with enabled allure-mocha reporter (2.0.0-beta.9) breaks test run with various errors:

Uncaught TypeError: test.parent.titlePath is not a function or its return value is not iterable

      at AllureReporter.startCase (node_modules\allure-mocha\src\AllureReporter.ts:98:62)
      at MochaAllureReporter.onTest (node_modules\allure-mocha\src\MochaAllureReporter.ts:39:23)    

I looked into code, and noticed that for mocha 8 code:
const [parentSuite, suite, ...subSuites] = test.parent.titlePath();
must be updated to
const [parentSuite, suite, ...subSuites] = test.parent.fullTitle();
Which might break backward compatibility. But report can be generated now.

But after updating there are still issue with using .createStep:
const step = allure.createStep(stepName, () => {}

allure cannot be imported since it is undefined.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

Demo repo can be taken from : https://github.com/Heisenbug-2021-api-testing-workshop/heisenbug-2021-api-testing-workshop-Xotabu4

Update .mocharc.js - add parallel: true to reproduce

What is the expected behavior?

What is the motivation / use case for changing the behavior?

Please tell us about your environment:

Allure version	2.7.0
Test framework	mocha@latest
Allure adaptor	[email protected]
Generate report using	allure-server@latest

Other information

[sskorol]

@Xotabu4 I can reproduce it. The first set of issues are related to https://mochajs.org/#limited-reporter-api-for-third-party-reporters. We can workaround it by checking if corresponding functions exist in runtime. However, the issue with the undefined allure instance is much trickier. Not sure how to fix it yet. I need to understand how Mocha processes reporters in a parallel mode first.

[just-boris]

Cross posting my ideas from v1 repo: https://github.com/allure-framework/allure-mocha/issues/65#issuecomment-750886832

[jamesmortensen]

@just-boris is it possible to unarchive that allure-mocha repo to bring your comment over here? The comment isn't accessible since that repo is now archived.

If not, here is a script we adapted from the one you originally posted, and it works with tests written as ESM modules:

parallel-launcher.js:

/*

Although Mocha v8.0.0 introduces parallel execution, many reporters are not compatible,
and allure is one of them. As a workaround, we have launcher and runner scripts to execute
separate worker processes to run individual test files in parallel.

# Running the tests

$ npx mocha parallel-launcher.js && node get-results.js

At this time, there are no command line options. Edit the file directly to change settings.

- MAX_THREADS - The number of workers to spawn.
- testFiles - The glob pattern matcher of the test files (tests/*.js)


## Resources

- [Adapted from just-boris's launcher scripts](https://github.com/allure-framework/allure-mocha/issues/65#issuecomment-820274879)
- [Using Mocha Programmatically](https://github.com/mochajs/mocha/wiki/Using-Mocha-programmatically)
- [Mocha Source Code](https://github.com/mochajs/mocha)
*/

const MAX_THREADS = 7;

const glob = require('glob');
const prepend = require('prepend-transform').pt;
const rimraf = require('rimraf');

const testFiles = glob.sync('./tests/**/*.mjs');

async function run() {
    // using dynamic import since sindresorhus is converting all his modules to esm!
    // https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c
    const pLimit = (await import('p-limit')).default;
    const { execa } = await import('execa');

    const limit = pLimit(MAX_THREADS);

    const failedSpecs = [];

    let failedTestCaseCountsArray = await Promise.all(
        testFiles.map(testFile =>
            limit(async () => {
                const subProcess = execa('mocha', [testFile]);
                const prefix = `${testFile}: `;
                subProcess.stdout.pipe(prepend(prefix)).pipe(process.stdout);
                subProcess.stderr.pipe(prepend(prefix)).pipe(process.stderr);
                try {
                    await subProcess;
                    return { failed: 0, testFile: testFile };
                } catch (error) {
                    failedSpecs.push(testFile);
                    return { failed: error.exitCode, testFile: testFile };
                }
            })
        )
    );

    const failedCount = getFailedCount(failedTestCaseCountsArray);
    console.log('total failed test cases = ' + failedCount);
    await writeFailedCountToFile(failedCount);

    if (failedSpecs.length > 0) {
        throw new Error('Failed Test Results: \n' + getTestResults(failedTestCaseCountsArray));
    } else {
        console.info('All Tests Pass!');
    }
}

rimraf.sync('allure-results');
rimraf.sync('allure-report');

run().catch(async error => {
    console.error(error);
});


function getTestResults(failedTestCaseCountsArray) {
    return failedTestCaseCountsArray.map((elem, ind, arr) => { return elem.testFile + ': ' + elem.failed; }).join('\n')
}

function getFailedCount(failedTestCaseCountsArray) {
    return failedTestCaseCountsArray.reduce((acc, res, index, arr) => {
        acc += res.failed;
        return acc;
    }, 0);
}

async function writeFailedCountToFile(failedCount) {
    const { execa } = await import('execa');
    const _process = execa('sh', ['./bin/generate_results.sh', failedCount]);
    _process.stdout.pipe(process.stdout);
}

The following two scripts deal with the fact that, when running mocha in parallel this way, the exit code does not return non-zero when test cases fail: mochajs/mocha#3893. We get around that by writing the number of failed cases to a file and then running a separate script that then reads that number and returns an exit code which CI systems can then use to flag steps as passed/failed as needed:

generate_results.sh:

echo $1 > /tmp/launcher_results.txt

get-results.js:

const fs = require('fs');

const resultsTxt = fs.readFileSync('/tmp/launcher_results.txt', 'utf-8');
const results = parseInt(resultsTxt);
console.log(results)

if(results !== 0)
   process.exit(results);