Make nyc works with mock-fs

[thangngoc89]

I'm using mock-fs in my tests for mocking fs.

All tests passed when not using nyc. But when I run coverage with nyc. Both nyc and test fail. The process is killed.

Here is an example:

// test file
import test from "ava"
import mock from "mock-fs"
import compiler from "../compile-template"

test.afterEach(() => {
  mock.restore()
})

test("compile template with lodash.template", async (t) => {
  mock({
    "/some/path": "foo <%= bar %>",
  })

  const result = await compiler("/some/path", {
    bar: "bar",
  })

  t.is(result, "foo bar")
})

$ nyc ava src/_utils/service-worker/__tests__/compile-template.js

  ✖ compile template with lodash.template failed with "ENOENT: no such file or directory, open '/some/path'"
Uncaught Exception: src/_utils/service-worker/__tests__/compile-template.js
  Error: ENOENT, no such file or directory '/home/khoa/web/statinamic/.nyc_output/28496.json'
    Binding.<anonymous> (/home/khoa/web/statinamic/node_modules/mock-fs/lib/binding.js:287:15)
    maybeCallback (/home/khoa/web/statinamic/node_modules/mock-fs/lib/binding.js:41:17)
    Binding.open (/home/khoa/web/statinamic/node_modules/mock-fs/lib/binding.js:274:10)
    Object.fs.openSync (/home/khoa/web/statinamic/node_modules/mock-fs/node/fs-5.0.0.js:584:18)
    Object.fs.writeFileSync (/home/khoa/web/statinamic/node_modules/mock-fs/node/fs-5.0.0.js:1224:33)
    NYC.writeCoverageFile (/home/khoa/web/statinamic/node_modules/nyc/index.js:323:6)
    EventEmitter.onExit.alwaysLast (/home/khoa/web/statinamic/node_modules/nyc/index.js:297:11)
    emit (/home/khoa/web/statinamic/node_modules/signal-exit/index.js:68:11)
    processEmit [as emit] (/home/khoa/web/statinamic/node_modules/signal-exit/index.js:143:5)

[jamestalmage]

mock-fs is problematic. You can use mountfs to get around it. See avajs/ava#665

[thangngoc89]

Thank you. I'll give it a go

[jamestalmage]

Even following the linked instructions, nyc causes the repo to fail.

See https://github.com/thangngoc89/nyc-mock-fs

I'm stumped. @novemberborn ?

[jamestalmage]

Seems promise-fs is messing things up somehow, even though the file system is fully mocked.

See thangngoc89/nyc-mock-fs#1.

Switching from promise-fs to just pify and fs fixes the problem.

[novemberborn]

fs-promise uses mz which tries to load graceful-fs. Unfortunately nyc has a transitive dependency on graceful-fs as well. This means fs-promise ends up binding against the unmocked graceful-fs.

The pify approach is late-binding against fs.readFile, thus it gets to use the actual mocked filesystem.

I'd say this isn't an issue with nyc. It's other libraries doing strange things.

[jamestalmage]

Thanks for the in depth research @novemberborn.

I'd say this isn't an issue with nyc. It's other libraries doing strange things.

Agreed.

[bcoe]

This is definitely a danger with nyc having any library dependencies, I wonder if there should be a step where we clear the require-cache -- Isaacs told me this can be risky mind you, creating more opportunities for memory leaks.

[novemberborn]

We might be better off reducing the dependencies we introduce into the child processes, possibly rewiring (à la proxyquire) any remaining dependencies so their bindings survive mocking.

[jamestalmage]

Why not just shrinkwrap everything?

[novemberborn]

@jamestalmage assuming npm doesn't dedupe the dependency shrinkwrap I guess that would work for nyc's dependency tree, aside from when the standard modules are mocked.

[bcoe]

@novemberborn @jamestalmage perhaps we start bundling dependencies for nyc ... as much as I'm not a fan of this approach, it does stop singletons from breaking.

[jamestalmage]

What is the downside? I understand there are implications for a normal library, but for one that is basically guaranteed only to be used as a dev-dependency I think those downsides are mitigated somewhat.

We probably should seek outside opinions.

@isaacs @sindresorhus

[sindresorhus]

ESLint recently tried bundling, but encountered an issue with bundledDependencies and npm@2: eslint/eslint#5013

[bcoe]

@thangngoc89 @novemberborn @jamestalmage give this branch a shot on a few of your repos:

npm i nyc@bundle-test

I've started bundling dependencies, it should also fix the issue with node 5.8.0.

[thangngoc89]

@bcoe nice. Test passed in https://github.com/thangngoc89/nyc-mock-fs . I'll take another look at it with a real app at the end of the day