Support npm 7 #128

vweevers · 2020-10-17T15:54:15Z

Closes #108.

Instead use npm_config_* which works as far back as npm > 0.1.7. It translates command line flags to environment variables, where: --foo becomes npm_config_foo='true' --no-foo becomes npm_config_foo='' --foo=false becomes npm_config_foo=''

bin.js

test/skip-test.js

util.js

test/rc-test.js

bin.js

vweevers · 2020-10-18T12:39:45Z

Wrote npm-script-context to help debug the various cases. Oddities in npm 7:

When installing a github dependency (e.g. github:foo/bar), its install scripts run twice: after the repository is cloned to npm's global cache (cacache) and after copying from cache to local node_modules. In either case, there's no .git directory that we could detect.
When installing a transitive dependency, the output of its install script is swallowed regardless of --loglevel
env.npm_package_from is never defined, I tried various scenarios. I do however see npm_package_resolved=git+ssh://[email protected]/foo/bar.git#abc (but not in the cache case of 1)

I.e. when running `npm install` in the working directory of a project that uses prebuild-install, you must now run a more explicit `npm install --build-from-source`. This is necessary to support npm 7, where we do not have enough information to automatically determine it and would erroneously skip downloads on normal installs.

lovell · 2020-10-20T15:39:47Z

Thank you for tackling this Vincent.

https://github.com/npm/cli/blob/latest/CHANGELOG.md#all-lifecycle-scripts suggests we should expect npm_package_resolved (there's no mention of npm_package_from), which I think on its own is enough to recreate the existing _from based logic. If it's not being made available in some cases then that could be a bug.

vweevers · 2020-10-20T16:20:33Z

which I think on its own is enough to recreate the existing _from based logic.

Aye, that's what I ended up doing in this PR. With npm_package_resolved we can detect a git repository (npm i user/foo or npm i git+file://...) but we cannot detect a standalone install (cd foo && npm i). I removed that behavior (see last commit).

lovell · 2020-10-20T17:37:06Z

The lack of standalone install might be a bit annoying for local development e.g. can no longer just run npm i to force compilation, but I agree that the default behaviour should be to download rather than compile.

A possible workaround for local development might be to patch-bump the package.json version in a commit straight after running npm publish to ensure prebuilt binaries aren't found (although this would prevent the use of prebuild-ci).

vweevers · 2020-10-23T06:32:28Z

@lovell is that a blocker for you, and why is npm i --build-from-source not sufficient?

lovell · 2020-10-23T08:30:37Z

@vweevers No blocker, it's only a small change to my development workflow. The benefits to end users far outweigh this, which is where our focus should lie.

.travis.yml

rc.js

util.js

vweevers · 2020-10-23T10:27:56Z

Not squashing this, because the individual commit descriptions (especially ddb8f69) are useful.

vweevers · 2020-10-23T10:31:22Z

I forgot @ralphtheninja released a new version in the mean time, I should have rebased, commits are now out of order. Sorry.

vweevers · 2020-10-23T11:03:50Z

I think in this case a rebase and force push to master is acceptable. Otherwise people will be confused why commits with breaking changes are listed before 5.3.6 (although we have a changelog now but not all folks read that).

Do you agree @lovell @ralphtheninja?

vweevers · 2020-10-23T11:43:05Z

Done

See lovell/sharp#2419 and prebuild/prebuild-install#128

Sharp 0.26.3 updates prebuild-install to ^6.0.0 which fixes an incompatibility with NPM 7. See lovell/sharp#2419 and prebuild/prebuild-install#128

vweevers added 4 commits October 17, 2020 14:55

Change rc-test to npm integration test

9cec98b

Test npm conditions that skip downloads

4ab06c6

Read env.npm_package_from for npm 7

c24e6f5

Remove use of env.npm_config_argv for npm 7

80556b5

Instead use npm_config_* which works as far back as npm > 0.1.7. It translates command line flags to environment variables, where: --foo becomes npm_config_foo='true' --no-foo becomes npm_config_foo='' --foo=false becomes npm_config_foo=''