feat!: add @vitest/coverage-v8 package

[AriPerkkio]

• Adds new coverage provider package @vitest/coverage-v8. For end-users it should provide identical coverage results as @vitest/coverage-c8.
  • This package does not implement the coverage options excludeNodeModules, allowExternal and src. I think we can implement these if needed, but at the moment even the @vitest/coverage-istanbul doesn't have these.
• Adds deprecation warning to @vitest/coverage-c8. This package is still around but will be removed in future breaking-change releases.
• Changes default coverage.provider from 'c8' to 'v8'

Migration

npm uninstall @vitest/coverage-c8
npm install --save-dev @vitest/coverage-v8

// vitest.config.ts
export default defineConfig({
  test: {
    coverage: {
-      provider: 'c8'
+      provider: 'v8'

Troubleshooting

• If your coverage report is completely empty, please check that your test.root does not exclude the source files. The root is now resolved correctly, identical to @vitest/coverage-istanbul: Provider v8 does not show test coverage #3555

Motivation for new package

The @vitest/coverage-c8 has always had to utilize some private APIs of c8. This has been required since c8 isn't really intended for libraries as it's rather for standalone usage. Non-breaking changes in c8 like bcoe/c8#469 can break @vitest/coverage-c8 without major version bumps.

The @vitest/coverage-c8 is using c8 only for merging, processing and converting the V8 coverage reports, and for creating the final reports. Code instrumentation moved from expensive process.env.NODE_V8_COVERAGE to node:inspector usage already in #2786.

Now with @vitest/coverage-v8 we don't use c8 package at all. New dependencies compared to @vitest/coverage-c8 are added to preserve the functionality:

• bunch of istanbul packages to process coverage results and create reports. Same packages are used by @vitest/coverage-istanbul already.
• @bcoe/v8-coverage for merging V8 coverage reports - this package is also used by c8 and jest
• v8-to-istanbul for converting the coverage reports from V8 to istanbul format

Enables (or at least makes it easier to implement) #3251 as now we have more control over c8.

BREAKING CHANGES:

• @vitest/coverage-istanbul requires latest vitest version. Some utilities were moved to vitest package for sharing them with the new v8 provider.
• Default coverage.provider is now 'v8' instead of 'c8'

Next steps

Once this PR is released we can:

• Remove @vitest/coverage-c8 package completely. The old releases will still remain in NPM registry. Latest release has deprecation warning in place.
• Remove all mentions of c8 from documentation

Testing:

Compare results between existing @vitest/coverage-c8 and the new @vitest/coverage-v8.

• test/coverage-test
• test/workspaces
• Manual testing with third party projects:
  • zustand: 100% coverage match
  • vuejs/core: 99% coverage match. Some conditional T extends U types are not considered as branches anymore. Earlier we didn't pass source maps for files containing only typings (not sure why), and let c8 resolve the source map. Now we correctly pass the source map from vite-node.
  • netlify/cli: 100% coverage match

[stackblitz]

Run & review this pull request in StackBlitz Codeflow.

[AriPerkkio]

Does the package name @vitest/coverage-c8 make any sense anymore? Maybe @vitest/coverage-v8 instead?

[dammy001]

Does the package name @vitest/coverage-c8 make any sense anymore? Maybe @vitest/coverage-v8 instead?

@AriPerkkio if changed, @vitest/coverage-c8 will be depreciated at first right?

[AriPerkkio]

if changed, @vitest/coverage-c8 will be depreciated at first right?

Yes, I think we would simply add a warning to be logged in its next minor/breaking-change-release and leave the package as is. This possible new package would be published as separate package with new name.

[AriPerkkio]

Moved all changes of @vitest/coverage-c8 package to new package @vitest/coverage-v8 instead. This PR does not remove anything c8 related - it only adds a deprecation warning message to @vitest/coverage-c8 initializer.

Small breaking change in @vitest/coverage-istanbul which now requires latest vitest version.

[antfu]

The direction SGTM 👍