[RRFC] support different --before policy per package prefix/pattern

[osher]

Motivation ("The Why")

Consider a corporate with a codebase with a high coverage and a mature CI.
Consider that the corporate's policy is to test on latest and greatest and ship latest and greatest to enjoy all security patches published by the community.
Now consider a counter requirement not to use 3rd-party versions that are too young, because they have not spent enough time in the market for their exploits to be noted.

This requires npm i to distinct between packages of the organization, who's new versions should be accepted as soon as they are published, in contrast to 3rd party open-source packages, who's versions should be X days old before it can enter a build (given that no known issues are detected on them).

This requires a further evolution over the npm i --before <date> option
We need npm to be able to distinct between corporate proprietary packages and the rest.

This could be using scopes, however, I meet too many codebases that still use prefixes which are not scopes, so I believe regex or at least prefixes to be a better solution.

It will also be useful to configure the behavior for not finding a version old enough. Some cases it would be preferable to fail, others to warn.

Example

Assuming we go with regex - then:

npm config set stabilityDays 7d
npm config set 'onOnlyTooNew' warn
npm config set '@my-corp/*.stabilityDays' 0
npm config set '@my-corp/*.onOnlyTooNew' info
npm config set package-that-gave-problems-before.stabilityDays 90
npm config set 'package-that-gave-problems-before.onOnlyTooNew' error

How

Current Behaviour

The current behavior does not distinct between packages.
There is a --before <date> switch, and it will reject too new versions - but it will know to let corporate packages to pass, inhibiting quick responses.

Desired Behaviour

Consider the example above - then:

• by default - versions newer than 7 days are ignored.
• packages who's name start with @my-corp/ (actually match the pattern `/@my-corp/.*/) can pass through
• package-that-gave-problems-before only versions 90 days or older

References

• [Feature] ability to install packages with at least X days since publication cli#4494

[ljharb]

An alternative for such a corporation is to configure their internal registry to “wait” N days until allowing a public package to be installable, which would require no changes in npm.

[osher]

I tried to find such config for jfrog because that's what the team I'm consulting now uses, but did not find a way to do that.

As for gitlab package registry - ATM it does not know to be an upstream at all... 😛

I don't know about nexus. Currently I see more teams leaving it for jfrog then teams leaving jfrog for nexus.

Here's an issue I started for verdaccio:
verdaccio/verdaccio#3029

I think that since not every team has their own registry, an npm-client setting would be both very powerful and useful 💪

So far I've been a strong proponent of npm, but I know the discussion will eventually get to the rest of the npm-clients out there.

If the implementation relies more on a server logic - it will make it easier to the rest of the clients to jump on board. However, the server should not keep user-settings, the client should store them locally and pass them.

On the other hand - if the entire logic is client side - then it means the upstream server won't matter and the functionality will work immediately for all teams. Mmm. donno. we can go both ways.

[ljharb]

I’m more arguing that since the use case is uncommon - waiting a length of time doesn’t increase security, and if such a practice became common then it would just delay discovery of vulnerabilities until after that time anyways - it should be implemented in your private registry instead of in npm itself.

[osher]

I did not miss your argument, and I see where it comes from. However - npm already supports --before, and anything you said is true about --before as well, but yet, --before/enjoy-by is supported. 🤷

The feature request is not about if to wait or not.
The mechanism is there, and although it was a lead when it was introduced - a lot has happened since then, so it has a small gap to catch up. 😉

The feature request is to be able apply a different policy to proprietary packages than to 3rd party 👍

[ljharb]

The before feature is massively useful to go “back in time” and debug when a problem started occurring; it’s got nothing to do with security concerns or waiting to update. I’ve used it extensively in the years it’s existed for this purpose alone.

[zkochan]

I agree that the before feature is not super useful from security perspective but it is very useful for caching package metadata. The current resolution mode demands the highest version that satisfies the range, so we need to always request a new metadata file and there's no way to use a metadata file from a local cache.

I have added a new resolution mode to pnpm called "time-based". This resolution mode uses time to resolve versions and allows to leverage a local cache better. Related RFC: pnpm/rfcs#2.

This resolution mode works perfectly. I have been using it for about two months already. The only issue is that the abbreviated metadata returned by the npm registry doesn't currently include the time field. So, when pnpm uses this resolution mode, it needs to request the non-abbreviated metadata. This makes installation slower with cold cache. It would be great if npm could return the time field with the abbreviated metadata.