Enable/document code coverage for extension testing

[gregvanl]

From @gorkem on May 18, 2017 17:20

The way to use a code coverage framework such as istabuljs with extension tests should be explained on https://code.visualstudio.com/docs/extensions/testing-extensions

Copied from original issue: microsoft/vscode#26879

[hilleer]

Is it actually possible to run the tests with a coverage tool just yet?

[emilianoLeite]

@gregvanl @gorkem Does VSCode already allow integration with code coverage frameworks?

[buehler]

hmm... ping? :-)

[DonJayamanne]

There's a very good sample here https://github.com/codecov/example-typescript-vscode-extension

[hilleer]

@DonJayamanne thank you - I'll have a look at that for sure :-)

[DracotMolver]

I'm facing the same issue. I don't wanna have a boilerplate configuration to run my test and get a code coverage (I'm talking about the @DonJayamanne example). It would be just greate to do this:

"cover": "istanbul cover ./node_modules/vscode/bin/test"

[DanTup]

I managed to get this working for my extension, but it was a faff... Info is here:

https://github.com/Dart-Code/Dart-Code/blob/02c019bff3c742cedb190b3b7acc5fe8b472e30e/package.json#L684-L696

and here:

https://github.com/Dart-Code/Dart-Code/blob/02c019bff3c742cedb190b3b7acc5fe8b472e30e/test/test_runner.ts#L10-L12

I had to pre-instrument the files (since nyc doesn't seem to be able to inject itself when running tests because of how Code sits in the middle), manually save coverage from within the test files, manually re-map the JS to TS and also write empty coverage files so that un-executed files appear in the report!

[connor4312]

Istanbul is marked as deprecated. They recommend users move to nyc, and I got this running in an extension this morning pretty easily by adding the following to my test runner script:

const NYC = require('nyc');

// create an nyc instance, config here is the same as your package.json
const nyc = new NYC({ 
  cwd: join(__dirname, '..', '..', '..'), // in debugging sessions, the cwd seems to be unset
  reporter: ['text', 'html'],
  instrument: true,
  hookRequire: true,
  hookRunInContext: true,
  hookRunInThisContext: true,
});

nyc.createTempDirectory(); // create nyc' temp directory
nyc.wrap(); // hook into require() calls, etc.


runMyTests();

nyc.writeCoverageFile();
nyc.report(); // write out the reports!

[hilleer]

@connor4312 Nice. Thanks for sharing :)

[dcermak]

@connor4312 Thanks for sharing this snippet! Would you happen to have a more complete example? I've been trying to integrate your code into runTests.ts from the yo code example, but so far I was unable to get any code coverage reported at all (I get a report but it claims zero code coverage, despite the tests actually being run).

[connor4312]

@dcermak here you go, this is now open source: https://github.com/microsoft/vscode-pwa/blob/master/src/test/testRunner.ts

[dcermak]

@connor4312 Thank you, that is greatly appreciated!

[gregvanl]

Closing as a couple for samples and examples have been provided. Not sure we need to add documentation.