[BUG] npx does not attempt to get newer versions #2329

jsg2021 · 2020-12-11T16:54:16Z

Current Behavior:

I have been using npx <some-workflow-helper> over the past year to run the current version of a command without having to manage updating it. npm7 does not pull the latest version if I have run it once before... i suspect because its no longer a temp install...

Expected Behavior:

npx some-package always runs the latest published version.

Steps To Reproduce:

npx <test-package>
publish a new version of
npx <test-package>

Environment:

OS: Fedora 33
Node: 14.15.0
npm: 7.1.0

The text was updated successfully, but these errors were encountered:

ljharb · 2020-12-11T16:56:19Z

If it’s installed locally or globally, it won’t fetch a version. If you want to force that, add @latest

jsg2021 · 2020-12-11T17:49:43Z

If I never "install" it and only use npx <> to run it, on npm6 you would always get the latest without extra args. Also, it wasn't actually installed, it was only temporarily installed... after running it was discarded. We lost that on npm7.

jsg2021 · 2020-12-11T17:53:46Z

I should amend ... yes, if the command exists it should obviously prefer that one. However, a command installed by npx should not be considered "installed"

leafac · 2021-01-26T09:49:48Z

I ran into this issue as well (see #2329).

Here’s some more information I gathered:

If you don’t have a command installed, then npx installs the package into the npm cache (on my machine that cache lives in ~/.npm).
If you run npx <command>@latest then npx fetches the latest version. But the next time you run just npx <command> again it’ll get the older version.
One workaround to force the package to update is to destroy the cache: rm -rf ~/.npm.

I think the best solution would be to mimic the behavior of running commands installed via npm install --global:

Provide some sort of npx --udpate <command>.
If the command is in the cache and you run npx <command>@latest, then the next time you run npx <command> it should choose the latest.
If the command is in the cache and you run just npx <command> then it should not check if there are updates available. First because you may not want to use the latest version, but more importantly you may not want to slow down running the command by going to the registry look for newer versions.

jsg2021 · 2021-01-26T16:48:44Z

I think the best solution would be to mimic the behavior of running commands installed via npm install --global:

Provide some sort of npx --udpate <command>.

If the command is in the cache and you run npx <command>@latest, then the next time you run npx <command> it should choose the latest.

If the command is in the cache and you run just npx <command> then it should not check if there are updates available. First because you may not want to use the latest version, but more importantly you may not want to slow down running the command by going to the registry look for newer versions.

I respectfully disagree. Unless actually installed (globally, or locally) the cache should only be used if the current match matches, otherwise it should pull a new one (and in npm7's case prompt, unless --yes is present). The whole reason for npx is to run commands in the npm context that you may not have nor care to keep track of. (for example internal CI or workflow tools that you rather not spend time updating across all the runners.)

For your use case, you would be better off by installing the command and running with the --no-install flag.

leafac · 2021-01-31T13:27:56Z

You make good points. Thanks for reading my proposal and for the feedback.

The whole reason for npx is to run commands in the npm context that you may not have nor care to keep track of. (for example internal CI or workflow tools that you rather not spend time updating across all the runners.)

For me, the whole reason for npx is somewhat different. It’s being able to tell people: “If you have Node.js installed, go ahead and use my tool simply by running npx obs-cli (for example); no installation necessary.” That’s only half-true, of course, because npx is installing the thing, but it doesn’t feel like it because it’s just on the cache; it doesn’t even show up on npm --global ls. The reduced friction of trying something without having to “install” it is appealing to some people (myself included).

jsg2021 · 2021-01-31T17:01:38Z

@leafac I believe we are saying the same thing at this point.

However, where I think we differ is the point of updating. I believe the convenance of not installing should come with the burden of always getting the latest... only falling back to cache if the latest is cached.

leafac · 2021-01-31T22:32:31Z

You make a good point…

darcyclarke · 2021-02-02T06:13:09Z

I believe #2592 should fix this issue once landed & is a duplicate of, if not related to, #2395

@latest

pull latest packument when getting info about package to run, also install newer version if one is available and user has not specified a version (i.e. is defaulting to @latest) PR-URL: #2592 Credit: @wraithgar Closes: #2395 Closes: #2329 Reviewed-by: @darcyclarke

wraithgar · 2021-02-02T17:23:37Z

v7 should once again match v6 behavior in the next release (v7.5.2). Changing it further should probably involve an RFC discussion.

leafac · 2021-02-05T23:48:24Z

@jsg2021: It seems like you got your way 😃

SrBrahma · 2021-04-24T01:58:07Z

I am using npm 7.11.1. I still uses an outdated package of mine.

When using npx myPackage, it uses v1.2.2, using npx myPackage@latest, uses v1.2.4, as expected.

What the hell? I don't have it globally installed. I don't want to tell my users to append @latest when calling my command. I don't want to append @latest to my npx calls.

npx should always use the latest version of the package. Unless maybe if there is no connection and there is a cached version that may or not be the latest one.

SrBrahma · 2021-04-24T02:00:06Z

Maybe, leave the devs an option to allow or not for the package to be executed at older versions in package.json.

I chose a 3 letter name for my package to be easy to call. I really don't want to add @latest every time.

ljharb · 2021-04-24T04:31:17Z

As a user, I never want to install something I already have available. I would much rather type @latest every time then be forced to implicitly incur a download cost just because the author deigned to publish a new version.

If I wanted that, I'd npm uninstall the package, so that it was downloaded every time.

SrBrahma · 2021-04-24T05:11:02Z

As a user, I like to use the latest versions by default because I don't really bother waiting 3-30s for a new version (that won't always happen) that may fix a breaking bug I don't know about and I may fall into it or add a cool new feature, and really dislike having to add @latest every single time because I don't know if I already have that package cached in a lower version.

For me, as an user, seems way better to use @current (or even @c) if for some reason I've never had during those years of npx is around, want to use the outdated cached version.

If I really want to use regularly a package and at a specific version, I can just npm i -g it. The npx philosophy as I understand, is to run a code you don't need to have in your machine. Basically running codes from cloud into your PC.

If this is just a matter of personal choice, it should be configurable and it won't be bad to have a poll on what should be the default value.

If I wanted that, I'd npm uninstall the package, so that it was downloaded every time.

I don't get this. How could I npm uninstall something I don't have really installed? I run my package with npx. Just tried npm uninstall myPackage, and as expected, nothing was removed.

Also, wasn't it fixed in 7.5.2, AFAIK? What about not only this issue itself, but as I found out now, #2641 and facebook/create-react-app#10601?

This behaviour seems way too prone for user errors, like the issues above. People, like me, will have to find out that their npx call is not using the latest package version and that may be leading to unexpected results. I am glad that I added the require("../package.json").version to the error messages, else, I don't really know how much longer it would take me to find that out.

itstheandre · 2021-06-01T17:41:09Z

Hey there! I know this has been closed, but I am now facing this issue again.

I created a package a while ago ironlauncher and when i do npx ironlauncher i am getting version 0.9.6. However in npm the current version is something like 0.20+ already. I know i dont have ironlauncher installed globally or done via npm link.

So what is the reason for this?

Anyone knows how to fix it?

SrBrahma · 2021-06-01T18:26:51Z

I insist to the npm devs: Add a new option to the package.json like "npxCheckLatest: true". Being it true and this package being npx'cuted, it will check if the cached version is the latest available one and will fetch the new one if required. This will leave the choice to the developers of the package, so the desired and intended behaviour of the respective npx package will be reached.

ljharb · 2021-06-01T18:38:03Z

That seems like a choice that should be up to the consumers of the package only - just like it already is in a package.json dependencies entry.

itstheandre · 2021-06-01T18:57:28Z

@ljharb maybe i am misreading it, but how would you code this in a way that every consumer of my package gets the latest version?

I always assume that npx gives me the latest version if i dont have something installed globally

SrBrahma · 2021-06-01T19:03:00Z

That seems like a choice that should be up to the consumers of the package only - just like it already is in a package.json dependencies entry.

Again, I have my specific case: I ALWAYS want for my users to use the latest version of my package without having to type @latest. I chose a 3 letter package name to be quickly written and executed. It's a specific case that isn't impossible for other projects to have the same intention. In case, my comment above has 2 likes, so I ain't a crazy single one with that request.

It's a project boilerplate starter and every update is a improved / fixed version of the previous version. The philosophy of it is to always use the latest version, mainly because it's a project I made for myself. Like it or not, that's what I want for my package. If an user for some strange reason don't want to wait 5s to download a new version, he can just not use my project (and creating the boilerplate can take some minutes, so 5s to download the newer version is like nothing), I don't care.

So, the use case exists, I am one. How a project is used isn't up to the npm dev team, with all the respect. Your personal opinion shouldn't have priority over what the developers want for their packages.

ljharb · 2021-06-01T19:20:33Z

ftr I think the default behavior of npx should be to always grab latest - but either way I don't think a package author's intention is relevant here.

How a project is used is also not up to the package author - it's up to the consumers.

SrBrahma · 2021-06-01T19:56:01Z

ftr I think the default behavior of npx should be to always grab latest - but either way I don't think a package author's intention is relevant here.

How a project is used is also not up to the package author - it's up to the consumers.

It's up to the consumers to use or not a package that its philosophy is to use the latest version. I am both a regular user and a dev of my package. I already explained my use case, it exists and it's valid. It's simple, as an user I always want to use the latest version of this package when using npx without having to type @latest every single time. There isn't such an option for that. As a dev, I want all my users to be able to do that. If they don't like it, they can just don't use it at all.

ljharb · 2021-06-01T20:48:00Z

Such an option would only work for npx tho, and npx foo is supposed to be conceptually identical to npm install -g foo (or whatever the package name is) and then foo.

yw662 · 2021-06-18T01:50:00Z

and npx foo is supposed to be conceptually identical to npm install -g foo

Actually not. The wanted behavior of npx foo, should be conceptually identical to:

foo, with the package already installed globally or locally,
otherwise,
npm install foo, then foo, then npm uninstall foo, or uninstall conceptually and hopefully the cache can be useful.

It is important to do the uninstall step. npx should keep it clean as if it never runs, except the side effects caused by the command itself. People expect npx to work this way.

ljharb · 2021-06-18T05:14:12Z

@yw662 "conceptually identical" encompasses the steps you described.

yw662 · 2021-06-18T05:26:17Z

But 'npm i -g' would introduce a side effect that 'npx' should not. It is not identical.

jsg2021 · 2021-06-18T05:28:44Z

I think this issue should probably be locked. (It's already closed)

npx is operating as expected (as far as I can tell)

Bugs happen.

This issue was originally to document a perceived departure from v6 to v7. That has been clarified and rectified.

TomasHubelbauer · 2021-06-19T14:52:58Z

Seeing as the issue is not locked, yet, I'd like to add my own comment in support of NPX always installing the latest version of the package no matter what. The current behavior of being stuck on whatever version happened to be the latest the first time the user has run npx package seems really odd to me. The user is not choosing a specific version when using npx package the first time they run it - they want the latest. So why keep reusing that version? It is not special just because it happened to be latest the first time the user ran npx command. What is special is the actual current latest version. I'd like to see NPX always execute that one.

jsg2021 · 2021-06-20T02:20:00Z

npx is designed to always pull latest if the command is not already present. If you are getting an old one when running the command you are experiencing a bug.

SrBrahma · 2021-07-04T00:03:27Z

For other devs, here is my implementation to always use the latest version in my cli program:

const spinner = ora().start('Ensuring latest version');
const latestVer = await latestVersion('gev');
if (compareSemver(VERSION, latestVer) === -1) {
  spinner.info(`The current version of gev [${chalk.keyword('brown')(VERSION)}] is lower than the latest available version [${chalk.yellow(latestVer)}]. Recalling gev with @latest...`);
  const rawProgramArgs = process.argv.slice(2);
  await execa('npx', ['gev@latest', '--no-check-latest', ...rawProgramArgs], { env: {
    npm_config_yes: 'true', // https://github.com/npm/cli/issues/2226#issuecomment-732475247
  } });
  return;
} else { // Same version. We are running the latest one!
  spinner.succeed();
}

Anyway, there is indeed a bug there.

Running npx gev@latest won't update the cached version. Just tested with my program. Running npx gev after npx gev@latest will still use the older cached version.

(the auto @latest wasn't present in the 1.x.x)

npm cache clear --force is not fixing this.

edit: I removed ~/.npm/_npx and npx gev -v is now returning the @latest one.

yw662 · 2021-07-04T00:40:17Z

#3430 This may solve some issues I guess. Anyone tried it after version 7.18.0 ?

SrBrahma · 2021-07-04T03:02:56Z

#3430 This may solve some issues I guess. Anyone tried it after version 7.18.0 ?

My comment before happened on npm 7.15.x. The same happened now on 7.19.1.

itstheandre · 2021-07-06T11:02:00Z

For other devs, here is my implementation to always use the latest version in my cli program:
const spinner = ora().start('Ensuring latest version');
const latestVer = await latestVersion('gev');
if (compareSemver(VERSION, latestVer) === -1) {
  spinner.info(`The current version of gev [${chalk.keyword('brown')(VERSION)}] is lower than the latest available version [${chalk.yellow(latestVer)}]. Recalling gev with @latest...`);
  const rawProgramArgs = process.argv.slice(2);
  await execa('npx', ['gev@latest', '--no-check-latest', ...rawProgramArgs], { env: {
    npm_config_yes: 'true', // https://github.com/npm/cli/issues/2226#issuecomment-732475247
  } });
  return;
} else { // Same version. We are running the latest one!
  spinner.succeed();
}
Anyway, there is indeed a bug there.

Running npx gev@latest won't update the cached version. Just tested with my program. Running npx gev after npx gev@latest will still use the older cached version.

(the auto @latest wasn't present in the 1.x.x)

npm cache clear --force is not fixing this.

edit: I removed ~/.npm/_npx and npx gev -v is now returning the @latest one.

The workaround is quite smart. I think ill implement this for my cli then.

SrBrahma · 2021-07-06T17:58:56Z

Just for your information, the VERSION in imported/required from the ../package.json Em ter, 6 de jul de 2021 08:02, André de Albuquerque < ***@***.***> escreveu:

…

For other devs, here is my implementation to always use the latest version in my cli program: const spinner = ora().start('Ensuring latest version');const latestVer = await latestVersion('gev');if (compareSemver(VERSION, latestVer) === -1) { spinner.info(`The current version of gev [${chalk.keyword('brown')(VERSION)}] is lower than the latest available version [${chalk.yellow(latestVer)}]. Recalling gev with @latest...`); const rawProgramArgs = process.argv.slice(2); await execa('npx', ***@***.***', '--no-check-latest', ...rawProgramArgs], { env: { npm_config_yes: 'true', // #2226 (comment) } }); return;} else { // Same version. We are running the latest one! spinner.succeed();} Anyway, there is indeed a bug there. Running npx ***@***.*** won't update the cached version. Just tested with my program. Running npx gev after npx ***@***.*** will still use the older cached version. [image: image] <https://user-images.githubusercontent.com/28177550/124369538-07665a80-dc43-11eb-906f-bc5127498f0f.png> (the auto @latest <https://github.com/latest> wasn't present in the 1.x.x) npm cache clear --force is not fixing this. edit: I removed ~/.npm/_npx and npx gev -v is now returning the @latest <https://github.com/latest> one. The workaround is quite smart. I think ill implement this for my cli then. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2329 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGW7JDVVP4KC7YGITISU3J3TWLPDFANCNFSM4UW5JAWQ> .

iamjaredwalters · 2021-07-28T16:54:42Z

This issue still appears to be present in 7.19.0. Running npx myPackage -V confirms an outdated version of the package is being preferred by npx, rather than the latest version.

wraithgar · 2021-07-28T17:13:54Z

The bug as written has been fixed.

With a clean cache:

~/D/npm $ npx [email protected] -v
Need to install the following packages:
  [email protected]
Ok to proceed? (y) 
15.0.8
~/D/npm $ npx tap -v
Need to install the following packages:
  tap
Ok to proceed? (y) 
15.0.9

I am going to lock this issue because further comments are not going to get anyone's problems troubleshooted. If you feel there is a bug, please open a new issue but please be aware that if you have a package globally installed, npx will find and use that before trying to install something else, and it will not automatically fetch the latest version in that case.

jsg2021 added Bug thing that needs fixing Needs Triage needs review for next steps Release 7.x work is associated with a specific npm 7 release labels Dec 11, 2020

leafac mentioned this issue Jan 26, 2021

[QUESTION] How do I update a package in the cache? npm/npx#93

Closed

darcyclarke removed the Needs Triage needs review for next steps label Feb 2, 2021

darcyclarke assigned wraithgar Feb 2, 2021

darcyclarke added this to the OSS - Sprint 23 milestone Feb 2, 2021

darcyclarke added the Duplicate duplicate of another, existing issue label Feb 2, 2021

wraithgar closed this as completed Feb 2, 2021

darcyclarke mentioned this issue Feb 2, 2021

Release v7.5.2 #2597

Merged

ruyadorno mentioned this issue Feb 2, 2021

deps: upgrade npm to 7.5.2 nodejs/node#37191

Closed

wraithgar mentioned this issue Feb 12, 2021

[BUG] NPX uses cached GIST instead of latest #2641

Closed

mrmckeb mentioned this issue Feb 25, 2021

"npx create-react-app my-app" stopped working in 4.0.3 facebook/create-react-app#10601

Open

lastmjs mentioned this issue Jun 29, 2021

v0.3.0 sudograph/sudograph#55

Closed

85 tasks

npm locked as resolved and limited conversation to collaborators Jul 28, 2021

[BUG] npx does not attempt to get newer versions #2329

[BUG] npx does not attempt to get newer versions #2329

Comments

jsg2021 commented Dec 11, 2020

Current Behavior:

Expected Behavior:

Steps To Reproduce:

Environment:

ljharb commented Dec 11, 2020

jsg2021 commented Dec 11, 2020

jsg2021 commented Dec 11, 2020

leafac commented Jan 26, 2021

jsg2021 commented Jan 26, 2021

leafac commented Jan 31, 2021

jsg2021 commented Jan 31, 2021

leafac commented Jan 31, 2021

darcyclarke commented Feb 2, 2021

wraithgar commented Feb 2, 2021

leafac commented Feb 5, 2021

SrBrahma commented Apr 24, 2021

SrBrahma commented Apr 24, 2021

ljharb commented Apr 24, 2021

SrBrahma commented Apr 24, 2021 • edited Loading

itstheandre commented Jun 1, 2021

SrBrahma commented Jun 1, 2021

ljharb commented Jun 1, 2021

itstheandre commented Jun 1, 2021

SrBrahma commented Jun 1, 2021

ljharb commented Jun 1, 2021 • edited Loading

SrBrahma commented Jun 1, 2021

ljharb commented Jun 1, 2021

yw662 commented Jun 18, 2021 • edited Loading

ljharb commented Jun 18, 2021

yw662 commented Jun 18, 2021

jsg2021 commented Jun 18, 2021

TomasHubelbauer commented Jun 19, 2021

jsg2021 commented Jun 20, 2021

SrBrahma commented Jul 4, 2021 • edited Loading

yw662 commented Jul 4, 2021

SrBrahma commented Jul 4, 2021 • edited Loading

itstheandre commented Jul 6, 2021

SrBrahma commented Jul 6, 2021 via email

iamjaredwalters commented Jul 28, 2021 • edited Loading

wraithgar commented Jul 28, 2021

SrBrahma commented Apr 24, 2021 •

edited

Loading

ljharb commented Jun 1, 2021 •

edited

Loading

yw662 commented Jun 18, 2021 •

edited

Loading

SrBrahma commented Jul 4, 2021 •

edited

Loading

SrBrahma commented Jul 4, 2021 •

edited

Loading

iamjaredwalters commented Jul 28, 2021 •

edited

Loading