Convert json output into test2json output format

[stotz89]

Hello,
I know there are already a couple of issues and some discussions related to that topic however I wanted to ask this question. Since v2 Ginkgo produces its own json format as an output of the test results.

We are using Sonarqube to report back test coverage and test reports and unfortunately, at least as far as I know, Sonar expects the test reports in the test2json format produced by go test:

Path to test execution reports generated by Go with '-json' key, available since go1.10 (e.g.: go test -json > test-report.out).

One the one hand we could only use go test to get the desired outputs for test reports & coverage however on the other hand the output Ginkgo produces is just much more detailed that we also would like to see (in our CI pipeline). Therefore, our mitigation currently is to execute both steps (go test & ginkgo) to have the best of both but as said, should only be a mitigation.

So, my questions would be:

1. Someone else have already faced & solved this issue? Could a ReportAfterSuite convert the report back into the test2json format?
2. Would it be a feasible that Ginkgo can bring back the support for a test2json output format?

[onsi]

hey there @stotz89

I don't have a lot of experience/context with the test2json format though looking through it it does seem to be oriented towards streaming test information vs generating the comprehensive final report. With that said, I could imagine custom code in ReportAfterSuite generating test2json formatted report that probably gets close enough. I could also imagine a simple post-processing tool that can take a ginkgo generated JSON file and produce a test2json formatted file.

Would it be a feasible that Ginkgo can bring back the support for a test2json output format?

Just to clarify - there never has been support for the test2json format. Ginkgo's JSON format is new to V2 and did not displace/replace anything prior to v2. You can actually run go test -json > test-report.out with a Gingko suite but you'll simply get one test in the report: the test function in the suite's bootstrap.

As for how to add support for this in Ginkgo I think I'd lean twoards something like a new function in the reporters package that looks like:

func GenerateGoTestJSONReport(report types.Report, filename string)  error {
    // walk report and generate test2json-compatible objects
    // JSON-encode the objects into filename
}

Then users who want a test2json formatted report could simply add:

ReportAfterSuite("test2json", func(report types.Report) {
   Expect( reporters.GenerateGoTestJSONReport(report, "path/to/report.json")).To(Succeed())
})

Would you be interested in implementing something like GenerateGoTestJSONReport?

[stotz89]

Hi @onsi,
thanks for getting back. Sounds good to me - let me see if I find the time to write such a function!