Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pnpmInstall: Throws strange exceptions: Solved with "--no-parallel" #302

Open
jschneider opened this issue Feb 6, 2024 · 3 comments
Open
Labels
question Further information is requested

Comments

@jschneider
Copy link

I get these exceptions when calling gradle pnpmInstall from my project root (with several sub projects configured for pnpm):

> Task :internal:closed:webapps:meistercharts.com:pnpmInstall FAILED
Caching disabled for task ':internal:closed:webapps:meistercharts.com:pnpmInstall' because:
  Caching has not been enabled for the task
Task ':internal:closed:webapps:meistercharts.com:pnpmInstall' is not up-to-date because:
  Output property 'nodeModulesDirectory' file /home/johannes/projects/com.cedarsoft.monorepo/internal/closed/webapps/meistercharts.com/node_modules/@astrojs/check has been removed.
  Output property 'nodeModulesDirectory' file /home/johannes/projects/com.cedarsoft.monorepo/internal/closed/webapps/meistercharts.com/node_modules/@astrojs/check/LICENSE has been removed.
  Output property 'nodeModulesDirectory' file /home/johannes/projects/com.cedarsoft.monorepo/internal/closed/webapps/meistercharts.com/node_modules/@astrojs/check/dist has been removed.
Starting process 'command '/home/johannes/projects/com.cedarsoft.monorepo/internal/closed/webapps/meistercharts.com/build/node/pnpm/pnpm-v8.15.1/bin/pnpm''. Working directory: /home/johannes/projects/com.cedarsoft.monorepo/internal/closed/webapps/meistercharts.com Command: /home/johannes/projects/com.cedarsoft.monorepo/internal/closed/webapps/meistercharts.com/build/node/pnpm/pnpm-v8.15.1/bin/pnpm install
Successfully started process 'command '/home/johannes/projects/com.cedarsoft.monorepo/internal/closed/webapps/meistercharts.com/build/node/pnpm/pnpm-v8.15.1/bin/pnpm''
Scope: all 12 workspace projects
../../../..                              | Progress: resolved 1, reused 0, downloaded 0, added 0
../../../..                              |    +2358 ++++++++++++++++++++++++++++
../../../..                              | Progress: resolved 2358, reused 288, downloaded 0, added 0
../../../..                              | Progress: resolved 2358, reused 1799, downloaded 0, added 0
../../../..                              | Progress: resolved 2358, reused 2352, downloaded 0, added 0
../../../..                              | Progress: resolved 2358, reused 2352, downloaded 0, added 0, done
 ENOENT  ENOENT: no such file or directory, unlink '/home/johannes/projects/com.cedarsoft.monorepo/node_modules/.pnpm/node_modules/tunnel-agent'

The exact error message changes every time:

 ERR_PNPM_LINKING_FAILED  Error: ENOTEMPTY: directory not empty, rmdir '/home/johannes/projects/com.cedarsoft.monorepo/node_modules/.pnpm/[email protected]/node_modules/shikiji/dist/langs'

 ERR_PNPM_LINKING_FAILED  Error: ENOTEMPTY: directory not empty, rename '/home/johannes/projects/com.cedarsoft.monorepo/node_modules/.pnpm/[email protected][email protected]/node_modules/postcss-calc_tmp_15783' -> '/home/johannes/projects/com.cedarsoft.monorepo/node_modules/.pnpm/[email protected][email protected]/node_modules/postcss-calc'

The exception does not happen if adding "--no-parallel".

@deepy
Copy link
Member

deepy commented Feb 13, 2024

Since caching is not enabled we can rule out Gradle writing to the directory
And unless your subprojects are installing to the same node_modules we can rule out the plugin as well

Which then sadly would leave pnpm, you can work around the issue by using a shared build service as a semaphore, limiting the number of installs that can run in parallel

But before going down that route, just to confirm, all the pnpm installs aren't writing to /home/johannes/projects/com.cedarsoft.monorepo/node_modules/ right?

@deepy deepy added the question Further information is requested label Feb 13, 2024
@jschneider
Copy link
Author

The pnpm install tasks are writing all to the same node_modules dir in the project root.
We have a pnpm "monorepo" with a pnpm-workspace.yaml

@bryanstern
Copy link

I'm running into this as well with a similar project set up. Here's my workaround that I added in my root build.gradle.kts.

// Ensures that only one PnpmInstallTask runs at a time. The build service acts as a semaphore.
 class EmptyBuildService : BuildService<BuildServiceParameters.None> {
     override fun getParameters(): BuildServiceParameters.None = TODO("Not yet implemented")
 }
 val service = gradle.sharedServices
     .registerIfAbsent("pnpmInstallService", EmptyBuildService::class) {
         maxParallelUsages = 1
     }
 subprojects {
     tasks.withType<PnpmInstallTask>().configureEach {
         usesService(service)
     }
 }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants