v2 doesn't work with Yarn 2 with enableGlobalCache

[ylemkimon]

🐛 Bug Report

Have you read the [Contributing Guidelines on issues]

Yes

To Reproduce

npx @docusaurus/init@next init my-website classic
cd my-website
yarn set version berry
echo "enableGlobalCache: true" >> .yarnrc.yml
yarn install
yarn build

Expected behavior

Build the project successfully.

Actual Behavior

5:12:27 AM: Error: Cannot find module './dependencies/ContextElementDependency'
5:12:27 AM: Require stack:
5:12:27 AM: - /opt/build/repo/.yarn/$$virtual/webpack-virtual-fff521a369/3/buildhome/.yarn_cache/cache/webpack-npm-4.44.1-977bccfd33-5.zip/node_modules/webpack/lib/ContextReplacementPlugin.js
5:12:27 AM: - /opt/build/repo/.yarn/$$virtual/@docusaurus-core-virtual-cc39d80259/3/buildhome/.yarn_cache/cache/@docusaurus-core-npm-2.0.0-alpha.61-54eb014e4b-5.zip/node_modules/@docusaurus/core/package.json
5:12:27 AM: Require stack:
5:12:27 AM: - /opt/build/repo/.yarn/$$virtual/webpack-virtual-fff521a369/3/buildhome/.yarn_cache/cache/webpack-npm-4.44.1-977bccfd33-5.zip/node_modules/webpack/lib/ContextReplacementPlugin.js
5:12:27 AM: - /opt/build/repo/.yarn/$$virtual/@docusaurus-core-virtual-cc39d80259/3/buildhome/.yarn_cache/cache/@docusaurus-core-npm-2.0.0-alpha.61-54eb014e4b-5.zip/node_modules/@docusaurus/core/package.json
5:12:27 AM:     at Function.Module._resolveFilename (internal/modules/cjs/loader.js:966:15)
5:12:27 AM:     at Function.external_module_.Module._resolveFilename (/opt/build/repo/.pnp.js:19397:54)
5:12:27 AM:     at Module._load (internal/modules/cjs/loader.js:842:27)
5:12:27 AM:     at Function.external_module_.Module._load (/opt/build/repo/.pnp.js:19252:36)
5:12:27 AM:     at Module.require (internal/modules/cjs/loader.js:1026:19)
5:12:27 AM:     at require (internal/modules/cjs/helpers.js:72:18)
5:12:27 AM:     at Object.<anonymous> (/opt/build/repo/.yarn/$$virtual/webpack-virtual-fff521a369/3/buildhome/.yarn_cache/cache/webpack-npm-4.44.1-977bccfd33-5.zip/node_modules/webpack/lib/ContextReplacementPlugin.js:8:34)
5:12:27 AM:     at Module._compile (internal/modules/cjs/loader.js:1138:30)
5:12:27 AM:     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)
5:12:27 AM:     at Module.load (internal/modules/cjs/loader.js:986:32)
5:12:27 AM:     at Module._load (internal/modules/cjs/loader.js:879:14)
5:12:27 AM:     at Function.external_module_.Module._load (/opt/build/repo/.pnp.js:19252:36)
5:12:27 AM:     at Module.require (internal/modules/cjs/loader.js:1026:19)
5:12:27 AM:     at require (internal/modules/cjs/helpers.js:72:18)
5:12:27 AM:     at Object.<anonymous> (/opt/build/repo/.yarn/$$virtual/@docusaurus-theme-classic-virtual-6301812ffe/3/buildhome/.yarn_cache/cache/@docusaurus-theme-classic-patch-30ba593d9b-5.zip/node_modules/@docusaurus/theme-classic/src/index.js:16:34)
5:12:27 AM:     at Module._compile (internal/modules/cjs/loader.js:1138:30)

Cause

Docusaurus uses importFresh to require presets and plugins.

From yarnpkg/berry#706 (comment):

The problem is that importFresh uses resolveFrom, which creates a whole new module (second parameter in _resolveFilename):

https://github.com/sindresorhus/resolve-from/blob/188ed7da119c788d98272852e8a71fd19195f6cf/index.js#L29-L33

The new module loses information, two in particular:

• the file that owns the resolution that we're requiring (ie the .eslintrc.yml file).
• the dependency tree that "owns" the resolved entry.

As a result of the second one, the PnP loader tries to regenerate it by finding the closest ancestor PnP hook (.pnp.js) for the issuer. That would usually do the trick, but since importFresh also lost the issuer then the loader fallbacks to finding the closest ancestor for the request itselfy.

Unfortunately, in the case of the global cache, the request is a file within the cache, and the closest ancestor doesn't exist (because the cache doesn't have a .pnp.js file). So the preset is loaded as if it wasn't covered by a PnP dependency tree, hence the "Cannot find module" message (PnP would throw something more detailed).

One last mystery: even if we're loading the preset through the regular Node resolver instead of the PnP resolver, why did the resolution failed? After all, as we can see in the error, we're resolving an absolute path. So it makes sense that it would fail to load its dependencies, but we can't even load it at all - what happens? The answer is that the Node resolver cheats:

https://github.com/nodejs/node/blob/v13.5.0/lib/internal/modules/cjs/loader.js#L136

In order to be faster, they use a special native and internal stat function that isn't exposed to userland code and, more importantly, doesn't support reading from Zip archives ... so when it tries to check whether the file exists, it has no idea that it needs to look inside the archive, and the Node resolution fails.

How to fix?

It's a bit difficult to say because frankly the whole require / require.resolve API is an incredible mess due to the fact that it deals directly with file paths rather than abstract tokens. This is a very bad design, and as a result it's very hard for everyone to know exactly how the API should be used in a perfectly generic way.

I'll need to think some more about this.

Your Environment

• Yarn 2 with the environment variable YARN_ENABLE_GLOBAL_CACHE=1

Reproducible Demo

https://github.com/ylemkimon/berry-docusaurus-netlify
https://app.netlify.com/sites/awesome-allen-96f580/deploys/5f3992ee969cb10007a179e2

[slorber]

Thanks, let's track this but currently don't know how to fix it 🤪

[merceyz]

Fixed in yarnpkg/berry#1851, update to the latest yarn and it should work :)

[slorber]

thanks

@ylemkimon I'm closing but we can reopen if this does not work for you