A SonarQube XML reporter for Cypress.

Generated XML reports are compliant with Generic Execution described in https://docs.sonarqube.org/latest/analysis/generic-test/

Why another one?

Since this Cypress issue: cypress-io/cypress#1495, spec filename are not available for reporters in a Cypress environnement.

[EDIT 02-14-2020]: despite the correction of this issue since Cypress v3.8.3 (see Release Notes), the problem is still there and the below reporters are still not working...

So this reporter provides a workaround (simplified as possible) in order to be able to generate these SonarQube reports.

A SonarQube test execution report is created for each Cypress spec file.

Other existing Mocha SonarQube reporters:

• danmasta/mocha-sonar
• mmouterde/mocha-sonarqube-reporter

Tested with Cypress

Take a look at the Actions tests matrix results: tested with Node.js v[12.x, 14.x, 16.x, 18.x, 20.x] and Cypress v[4.x, 5.x, 6.x, 7.x, 8.x, 9.x, 10.x, 11.x, 12.x, 13.x]

Compatibility with Cypress Component Testing

With the integration of the recent changes in #47, the reporter now supports Cypress component testing. Users can leverage the functionalities of cypress-sonarqube-reporter for both end-to-end tests and component tests conducted through Cypress.

Example

The following Cypress/Mocha spec...

describe("The root suite", () => {
    it("Test case #1 (must pass)", () => {
        expect(true).to.be.true;
    });
    describe("A sub suite", () => {
        it("Test case #2 (must pass)", () => {
            expect(true).to.be.true;
        });
        it("Test case #3 (must fail)", () => {
            expect(true).to.be.false;
        });
    });
    describe("Another sub suite", () => {
        xit("Test case #4 (must be skipped)", () => {
            expect(true).to.be.false;
        });
        it("Test case #5 (must raise an error)", () => {
            undefined.toString();
        });
    });
});

...will provide the following generated SonarQube report (with default options)

<!-- File: dist/cypress/integration/Sample.spec.js.xml -->
<?xml version="1.0" encoding="utf-8"?>
<testExecutions version="1">
  <file path="test/cypress/integration/Sample.spec.js">
    <testCase name="The root suite - Test case #1 (must pass)" duration="64"/>
    <testCase name="The root suite - A sub suite - Test case #2 (must pass)" duration="34"/>
    <testCase name="The root suite - A sub suite - Test case #3 (must fail)" duration="513">
      <failure message="AssertionError: expected true to be false">
        <![CDATA[AssertionError: expected true to be false
    at Context.runnable.fn (http://localhost:62294/__cypress/runner/cypress_runner.js:101081:20)
    at callFn (http://localhost:62294/__cypress/runner/cypress_runner.js:30931:21)
    at http://localhost:62294/__cypress/runner/cypress_runner.js:104009:28
    at PassThroughHandlerContext.finallyHandler (http://localhost:62294/__cypress/runner/cypress_runner.js:135962:23)
    at PassThroughHandlerContext.tryCatcher (http://localhost:62294/__cypress/runner/cypress_runner.js:139407:23)
    at Promise._settlePromiseFromHandler (http://localhost:62294/__cypress/runner/cypress_runner.js:137343:31)
    at Promise._settlePromise (http://localhost:62294/__cypress/runner/cypress_runner.js:137400:18)
    at Promise._settlePromise0 (http://localhost:62294/__cypress/runner/cypress_runner.js:137445:10)
    at Promise._settlePromises (http://localhost:62294/__cypress/runner/cypress_runner.js:137524:18)
    at Promise._fulfill (http://localhost:62294/__cypress/runner/cypress_runner.js:137469:18)
    at Promise._settlePromise (http://localhost:62294/__cypress/runner/cypress_runner.js:137413:21)
    at Promise._settlePromise0 (http://localhost:62294/__cypress/runner/cypress_runner.js:137445:10)
    at Promise._settlePromises (http://localhost:62294/__cypress/runner/cypress_runner.js:137524:18)
    at Promise._fulfill (http://localhost:62294/__cypress/runner/cypress_runner.js:137469:18)
    at Promise._resolveCallback (http://localhost:62294/__cypress/runner/cypress_runner.js:137263:57)]]>
      </failure>
    </testCase>
    <testCase name="The root suite - Another sub suite - Test case #4 (must be skipped)" duration="0">
      <skipped message="skipped test"/>
    </testCase>
    <testCase name="The root suite - Another sub suite - Test case #5 (must raise an error)" duration="488">
      <error message="TypeError: Cannot read property 'toString' of undefined">
        <![CDATA[TypeError: Cannot read property 'toString' of undefined
    at Context.<anonymous> (http://localhost:62294/__cypress/tests?p=test\cypress\integration\Sample.spec.js-285:37:17)]]>
      </error>
    </testCase>
  </file>
</testExecutions>

Sample

A sample project demonstrating the use of this reporter in a React App with Cypress and SonarQube is available in example folder

From Mocha tests result to SonarQube Generic Execution report

The following table explains the association between test states and the generated XML part:

Mocha test state	SonarQube Execution testCase child node
"passed"	none
"pending"	<skipped message="skipped test"/> short message is always "skipped message"
"skipped"	<skipped message="An error occurred during a hook and remaining tests in the current suite are skipped"/>
"failed" and test.err.name === "AssertionError"	<failure message="AssertionError: expected true to be false"><![CDATA[AssertionError: expected true to be false at ...]]></failure>
"failed" and test.err.name !== "AssertionError"	<error message="TypeError: Cannot read property 'toString' of undefined"><![CDATA[TypeError: Cannot read property 'toString' of undefined at ...]]></error>

Installing

In Node.js environnement, use your favorite command:

npm install --save-dev cypress-sonarqube-reporter

yarn add --dev cypress-sonarqube-reporter

Usage

As a single Cypress reporter

As described in Cypress documentation, single configuration:

With Cypress version >= v10.x

// File: cypress.config.js
const { defineConfig } = require('cypress')

module.exports = defineConfig({
	reporter: 'cypress-sonarqube-reporter',
	reporterOptions: {
    	// see "Reporter Options" section
	}
});

With Cypress version < v10.x

// File: cypress.json
{
	"reporter": "cypress-sonarqube-reporter",
	"reporterOptions": {
		// see "Reporter Options" section
	}
}

Using Cypress multiple reporters plugin

As described in Cypress documentation, multi configuration:

// File: cypress.json
{
	"reporter": "cypress-multi-reporters",
	"reporterOptions": {
		"reporterEnabled": "cypress-sonarqube-reporter, mochawesome",
		"mochawesomeReporterOptions": {
			// mochawesome options...
		},
		"cypressSonarqubeReporterReporterOptions": {
			// see "Reporter Options" section
		}
	}
}

Spec files update

EDIT: since v1.11.0, this code snippet is useless if you're using Cypress v6+.

The magic behind the scene is the use of Cypress.spec object (see Cypress documentation) that is only available on spec files (ie not on reporter scope), so the drawback of this workaround is to use the function specTitle(title: string) from specTitle.js instead of the suite title:

const specTitle = require('cypress-sonarqube-reporter/specTitle');

describe(specTitle('The root suite'), () => {
	...
});

This Cypress.spec object is only available since Cypress v3.0.2 (see Cypress changelog)

To avoid suite title pollution in other reporters (like the great mochawesome), make sure that cypress-sonarqube-reporter is the first one in the list.

Merging reports into a single report

Since v1.10.0, you could merge all the generated reports into a single report. It could ease the configuration of your SonarQube analysis.

This feature is only available on Cypress version >= 6.2.0, since the after:run plugin event does not exist on previous versions (see Cypress changelog).

The merge operation has to be configured in Cypress plugins:

// File: cypress/plugins/index.js
module.exports = (on, config) => {
	// https://docs.cypress.io/api/plugins/after-run-api
	on('after:run', (results) => {
		// /!\ don't forget to return the Promise /!\
		return require('cypress-sonarqube-reporter/mergeReports')(results);
	});
};

or if you need to specify plugin's options:

// File: cypress/plugins/index.js
module.exports = (on, config) => {
	// https://docs.cypress.io/api/plugins/after-run-api
	on('after:run', (results) => {
		// /!\ don't forget to return the Promise /!\
		return require('cypress-sonarqube-reporter/mergeReports')(results, {
			// see "Merge Plugin Options" section for all available options
			mergeFileName: 'another-name.xml'
		});
	});
};

Reporter Options

Name	Type	Default	Description
outputDir	string	"./dist"	folder name for the generated SonarQube XML reports, will be automatically created if not exist
preserveSpecsDir	boolean	true	specify if tests folders structure should be preserved
overwrite	boolean	false	specify if existing reporters could be overwritten; if false then an error is raised when reports already exist
prefix	string	""	file prefix for the generated SonarQube XML reports
useFullTitle	boolean	true	specify if test case should combine all parent suite(s) name(s) before the test title or only the test title
titleSeparator	string	" - "	the separator used between combined parent suite(s) name(s); only used if useFullTitle is true
useAbsoluteSpecPath	boolean	false	specify if the absolute path of a spec file should be written to the report
mergeOutputDir	string	<none>	folder name for the merged SonarQube XML report, will be automatically created if not exist. If not specified, outputDir is used
mergeFileName	string	"cypress-sonarqube-reports.all.xml"	merged SonarQube XML report name

Merge Plugin Options

Name	Type	Default	Description
reportsOutputDir	string	"./dist"	folder name for the generated SonarQube XML reports. If not specified, reporter options are used
mergeOutputDir	string	<none>	folder name for the merged SonarQube XML report, will be automatically created if not exist. If not specified, reportsOutputDir is used
mergeFileName	string	"cypress-sonarqube-reports.all.xml"	merged SonarQube XML report name

Issues & Enhancements

For any bugs, enhancements, or just questions feel free to use the GitHub Issues

Licence

This project is licensed under the terms of the MIT license.