-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] npx behavior in CI environments #1935
Comments
@ruyadorno mentioned in the slack conversation that this was an intentional breaking change, could someone link to the context for that decision? |
hi @BethGriggs thanks for letting us know 😄 It's worth mentioning that for non-interactive terminals we are going to patch (ref #1936)
|
as also mentioned in the slack channel: |
we published 7.0.0-rc.4 today that skips the prompt, but prints a warning, when run in a non-interactive environment. that way you're still alerted that something is being installed but we do not break in CI environments. for interactive environments the above workaround should do the trick. can you let us know if this resolves your issue @BethGriggs? |
root@localhost ~# npm install -g [email protected]
npm ERR! code MODULE_NOT_FOUND
npm ERR! Cannot find module 'agentkeepalive'
npm ERR! Require stack:
npm ERR! - /usr/local/lib/node_modules/npm/node_modules/make-fetch-happen/agent.js
npm ERR! - /usr/local/lib/node_modules/npm/node_modules/make-fetch-happen/index.js
npm ERR! - /usr/local/lib/node_modules/npm/node_modules/npm-registry-fetch/index.js
npm ERR! - /usr/local/lib/node_modules/npm/lib/utils/metrics.js
npm ERR! - /usr/local/lib/node_modules/npm/lib/npm.js
npm ERR! - /usr/local/lib/node_modules/npm/lib/cli.js
npm ERR! - /usr/local/lib/node_modules/npm/bin/npm-cli.js
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2020-10-10T03_17_28_890Z-debug.log |
I think the concern has been addressed to reduce the impact in CI environments, thanks! |
closing since the original issue is resolved. @gengjiawen if you're able to reproduce the problem you had updating, please do open another issue and we can discuss it there |
FWIW, I bumped into this in TravisCI: https://travis-ci.com/github/kentcdodds/advanced-react-hooks/builds/191260988 I think that @BethGriggs' original concern is still an issue 😬 |
hmmm 🤔 maybe we should also check for the environment using |
Hitting this too, it broken all our CI scripts... Why did you introduce a breaking change in a minor/patch release? For anyone having this issue on Circle CI, you can set a global environment variable to set the |
@FezVrasta the change came in the 7.0.0 Semver-Major change. We thought we fixed the CI issue by checking for TTY, but it seems there are edge cases we missed. Looks like #2047 is open to hopefully fix this for more CI environments |
we published 7.0.6 today that should hopefully resolve this. we now skip the prompt entirely if you appear to be running in a CI environment. let us know if you continue to have issues! |
Works for us now: https://github.com/testing-library/react-testing-library/pull/809/files Thank you for the quick fix! |
Question: when npx7 prompts and then user installs the missing package,
|
For the second question, |
@3cp to answer to the first question: the packages are not installed as a global npm package, each npx install is placed in a separated hash-named folder within the npm cache folder, ref: https://github.com/npm/cli/blob/latest/lib/exec.js#L269 You can look them up with: |
@ruyadorno thx for the detail. We are able to reset the npx cache now. Is there a reason for npx to just reuse the cache and not check latest published version (periodically if want to save traffic)? I would love to see npx to eventually bring up the cache to latest published version. |
npx prompt content: ``` Need to install the following packages: install-peerdeps Ok to proceed? (y) ``` Ref: npm/cli#1935
In case anyone else finds this useful, I've made clearing the npx cache into a gist: Content: # Ref: https://github.com/npm/cli/issues/1935#issuecomment-745561262
rm -rf $(npm get cache)/_npx/* @ruyadorno would there be any negative effects from running this? (aside from having to re-download packages you use with |
I believe that's ultimately a design choice. In the npm arguments standard it's quite easy to specify you want the latest published version by appending
I don't think so 🙃 I believe they won't even necessarily be re-downloaded since the npx algo still shares the same cache as the npm installer, thus it might have the package in the global cache (usually |
Designwise, npm typically chooses to automatically assume @latest unless the package is in your package.json or lockfile. none of these cases apply to npx, so i would absolutely assume that it always gets the latest. A cache is just an implementation detail, and any time deleting the cache provides any different behavior than “it has to repopulate the cache” seems like an objective bug to me. |
Ok, we'll go with that! 😅
Ah, ok - so my "npx cache clear" script needs to run |
maybe it's the naming that is confusing and this is not a "npx cache" but rather a npx space in which these projects get installed to 🤔
no no no, when doing a fresh install |
Got it. Just to be sure, what I'm trying to do is make the next run of the same command (after clearing the "cache", or whatever it should be called) install and use $ npx install-peerdeps --yarn --dev @upleveled/eslint-config-upleveled
install-peerdeps v2.0.3 Running $ rm -rf $(npm get cache)/_npx/*
$ npx install-peerdeps --yarn --dev @upleveled/eslint-config-upleveled
...
install-peerdeps v3.0.3 |
cool 😎 yeah, that's the expected result 👍 |
@ruyadorno i don’t think “npx installs to a persistent space” is the mental model of any of its users. The common understanding I’ve heard for years is that it installs it for the life of the command, and then deletes it. |
Yeah right, that's a great point - from the pov of someone aware of these internal mechanisms the mental model makes sense but not so much from the userland perspective. Maybe it makes up for a good RFC discussion? Maybe it's just a bug? 🤔 I think @wraithgar has been working on a related fix, maybe he'll have more context to add to this discussion. |
Yep I think the related fix I'm working on will mitigate most, if not all of this. There are three underlying bugs, one is general caching (still debugging this one to see if So ... expect fix(es) soon on this. I'll have a PR for npm ready once I suss out where we want to land on the caching issue itself. |
I consider it a bug, yes, and I'm glad to hear it's likely to be fixed :-) |
Here is the fix that handles the outdated versions of things running during This doesn't touch or affect the |
There's a problem with env var solution ( Luckily, in my case I can just use |
Summary on the current state of things: |
node 14 ships with npm 6; node 16+ ships with npm 8. nvm and npm do not have LTS, only node does. |
npx prompt content: ``` Need to install the following packages: install-peerdeps Ok to proceed? (y) ``` Ref: npm/cli#1935
This isn't really a bug, but raising it to discuss the potential impact of changes to
npx
in CI environments.Current Behavior:
npx -y mocha
works to get the automatic install behaviour, but that would require users to update their CI scripts.Also,
npx -y mocha
errors in npm@6 with:I think that would complicate the logic required in CI scripts when running across multiple Node.js/npm versions.
Just using Mocha as an example. In Node.js, we use npx in our GH Actions configuration, which I suspect would be broken when GH Actions updates to use a version of Node.js containing npm@7.
Expected Behavior:
Steps To Reproduce:
npx <module>
in a CI enironment.Environment:
/cc @MylesBorins @richardlau
The text was updated successfully, but these errors were encountered: