[Meteor 3] Unable to run bundle from "meteor build"

[zodern]

Reproduction:

meteor create --release 3.0-alpha.19 test
cd test
meteor build --directory ../test-bundle
cd ../test-bundle/bundle/programs/server

# make sure node and npm versions match the versions in test-bundle/bundle/star.json

npm install

npm install errors with:

npm ERR! code EACCES
npm ERR! syscall open
npm ERR! path /tmp/test-bundle/bundle/programs/server/npm-shrinkwrap.json
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, open '/tmp/test-bundle/bundle/programs/server/npm-shrinkwrap.json'
npm ERR!  [Error: EACCES: permission denied, open '/tmp/test-bundle/bundle/programs/server/npm-shrinkwrap.json'] {
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'open',
npm ERR!   path: '/tmp/test-bundle/bundle/programs/server/npm-shrinkwrap.json'
npm ERR! }
npm ERR! 
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR! 
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.

For some reason npm tries to write to npm-shrinkwrap.json which fails since Meteor has always created it as a read only file. Old versions of Meteor worked correctly. There might be an issue with the npm-shrinkwrap.json file used in Meteor 3, or this might be caused by using a newer version of npm (though I wouldn't expect npm to try to update the shrinkwrap.json in this scenario).

[parlin]

I am experiencing thie EACCESS error when trying to migrate an older app up to 2.13.1
I get the error when deplying on my webhost, but I get no error when running locally..

Would be great to hear if anyone has info on this. thank you

npm ERR! code EACCES
npm ERR! syscall open
npm ERR! path /build/f8c6d03b-508b-4b3d-ab38-3deb77404a6b/.app-build/bundle/programs/server/npm-shrinkwrap.json
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, open '/build/npm_/.app-build/bundle/programs/server/npm-shrinkwrap.json'
npm ERR! [Error: EACCES: permission denied, open '/build/npm_/.app-build/bundle/programs/server/npm-shrinkwrap.json'] {
npm ERR! errno: -13,
npm ERR! code: 'EACCES',
npm ERR! syscall: 'open',
npm ERR! path: '/build/f8c6d03b-508b-4b3d-ab38-3deb77404a6b/.app-build/bundle/programs/server/npm-shrinkwrap.json'

[harryadel]

Yup, I run into this problem too. @nachocodoner @leonardoventurini Will you please take care of this problem in the next release?

[leonardoventurini]

@harryadel, could it be perhaps related to the Node.js version? I think if you create a project in for Meteor 3 from a Meteor 2 default install, you would need to make sure the bundle for Meteor 3 is also installed for it to properly build with the right Node.js/NPM version. A potential fix would be to make sure Meteor 3 is installed by default in the docker image.

cc @nachocodoner

[leonardoventurini]

By the way, I am very close to getting the node/binary command to download the bundle, already found a way to validate it locally, now need to understand how downloading the bundle works etc and implement the actual solution. I might pause it to take care of other issues though as until now there have always been workarounds.

[nachocodoner]

I've reproduced this issue in my Docker environment, used for development. It doesn't occur on other setups like Ubuntu or Mac without containerization.

Granting write permissions to npm-shrinkwrap.json resolves the issue. I'm considering automating on meteor build command a chmod for this file in specific environments to address the problem. Any thoughts on this fix?

I'm aware this raises security concerns, as modifying npm-shrinkwrap.json is not recommended as direct and transit dependencies could change unexpectedly between build the app from docker or within other env. I'll investigate further why Docker environments attempt to modify this file during npm install.

@harryadel: Did you get this issue within a docker container? or in other circumstances?

[harryadel]

@nachocodoner I get this issue when I try to deploy the application using Scalingo. Let me know, we can further assist you.

[harryadel]

@nachocodoner I attempted to verify your claim and you're right. Seems to be building fine locally but gets messed up in docker environments due to permission issues which can be circumvented using ENV METEOR_ALLOW_SUPERUSER=true and running as root.

https://github.com/harryadel/simpletasks/tree/main

You shall find a docker file in root directory, run it like so: docker build -t my-meteor-3-app .

and it'd yield the following result:

[harryadel]

Granting write permissions to npm-shrinkwrap.json resolves the issue. I'm considering automating on meteor build command a chmod for this file in specific environments to address the problem. Any thoughts on this fix?

I'm aware this raises security concerns, as modifying npm-shrinkwrap.json is not recommended as direct and transit dependencies could change unexpectedly between build the app from docker or within other env. I'll investigate further why Docker environments attempt to modify this file during npm install.

Even if we were to drop the security concerns, it's a band aid solution at best as we don't know the root cause of the issue. What changed between 2.x and 3.x to cause such error?

[nachocodoner]

I have investigated this issue on a Mac Intel machine, outside of a Docker environment, and the problem persists.

Build a Meteor 2 project

Meteor 2.x uses [email protected] and [email protected].

1- Run meteor build --directory ../test-bundle.
2 - Navigate to ../test-bundle/bundle/programs/server and run npm install.

The npx-shrinkwrap.json file is automatically modified and the write permission is not set.

Using [email protected] and [email protected], the error from this PR occurs, even with a Meteor 2.x bundle.

Build a Meteor 3 project

Meteor 3.x uses [email protected] and [email protected].

1 - Run meteor build --directory ../test-bundle.
2 - Navigate to ../test-bundle/bundle/programs/server and run npm install.

The npx-shrinkwrap.json file attempts to be modified, throwing the error from this PR, and the write permission is not set.

Using [email protected] and [email protected], the npx-shrinkwrap.json file is modified and the lockfileVersion is downgraded.

If we rebuild the Meteor 3 app and set the write permission with chmod 755 npx-shrinkwrap.json, then the npx-shrinkwrap.json file is modified with the lockfileVersion respected.

Conclusion

• The issue occurs broadly, not just in Docker environments.
• The npx-shrinkwrap.json file gets modified regardless of the npm version.
• Old npm versions ([email protected]) modify the npx-shrinkwrap.json even without write permissions.
• New npm versions ([email protected]) throw a permission error when modifying the npx-shrinkwrap.json without having write permissions.

This issue seems related to a change in the npm installer on new versions used by Meteor 3.x by default. The npx-shrinkwrap.json file needs to be modified regardless. I didn't find a reference of this issue reported as breaking change on npm though.

Said this, I will implement the fix of automating the permission set as part of the meteor build command to ensure proper installation during npm install later.

[harryadel]

We managed to have a workaround to deploy our Meteor application thanks to your suggestion @nachocodoner 🙏

https://github.com/harryadel/nodejs-buildpack/blob/af7c0ea815c45432cadb9c0c27a1be1e764d2a6a/bin/meteor.sh#L270

For anyone using Scalingo set BUILDPACK_URL env var to https://github.com/harryadel/nodejs-buildpack in order to use the modified version

[nachocodoner]

Changes on the meteor build command implemented and tested here, #13219

chmod 644 npm-shrinkwrap.json is actually enough.

[harryadel]

Great work, thank you for your efforts 👏

[StorytellerCZ]

@zodern @harryadel is this resolved now?

[denihs]

This should be fixed now. I'm closing this issue, but we open it again in the future if necessary.