Several npm packages not updating

[mockdeep]

Not sure if this will be related to #1169, but I have several NPM packages that can be upgraded, but Dependabot doesn't seem to be catching them, even if I click "Bump now". If I run yarn upgrade jquery it does work, however.

[feelepxyz]

@mockdeep sorry for the neglect! Is this fixed? Looks like jquery is up to date and there are a few other dependencies that would be updated if you bumped now.

[mockdeep]

@feelepxyz I just updated my dependencies with Dependabot. It made a bunch of PRs, I merged them and ran it again, and it didn't come up with any more. However, there are still 2 packages that are out of date (plus one exotic package):

Package                           Current Wanted Latest Package Type    URL                                                          
@typescript-eslint/eslint-plugin  1.13.0  1.13.0 2.5.0  devDependencies https://github.com/typescript-eslint/typescript-eslint#readme
@typescript-eslint/parser         1.13.0  1.13.0 2.5.0  devDependencies https://github.com/typescript-eslint/typescript-eslint#readme
eslint-import-resolver-typescript 1.1.1   exotic exotic devDependencies JounQin/eslint-import-resolver-typescript#feat/resolve_dts   
Done in 3.03s.

[feelepxyz]

@mockdeep oh looks like we're not doing a good job of updating dependencies with peer dependencies.

@typescript-eslint/[email protected] requires @typescript-eslint/parser@^2.0.0 which is preventing both dependencies from updating to not break each other but we should be dealing with this better.

Would love to fix this but we're busy integrating Dependabot at GitHub so will be a while before we get some time for this. It's a tricky to get this logic right and we want to err on the side of caution and not update where it might break your project.

[mockdeep]

@feelepxyz sounds like a pretty tall order. One other thing I'm noticing is that it appears it upgraded eslint in a way that broke the peer dependency for those two libraries. Without them upgraded I get the following warnings:

warning " > @typescript-eslint/[email protected]" has incorrect peer dependency "eslint@^5.0.0".
warning " > @typescript-eslint/[email protected]" has incorrect peer dependency "eslint@^5.0.0".

[ahammond]

I saw something similar. Dependabot appears to have ignored all the @aws-cdk/* libraries in https://github.com/awslabs/aws-delivlib/blob/master/package.json#L37

[clareliguori]

Same here on @aws-cdk/* libraries. Dependabot will update the aws-cdk devDependency, but not the @aws-cdk/* dependencies. Example:
https://github.com/aws-samples/aws-reinvent-2019-trivia-game/blob/c36e4643f43acca7006c4cdbea7919698d9b3571/trivia-backend/infra/cdk/package.json
In this round of Dependabot PRs, I got a PR for aws-cdk upgrading from 1.15 to 1.24, but it never upgraded @aws-cdk/aws-certificatemanager or @aws-cdk/core from 1.15 for example. Usually I upgrade all of these at the same time.

To @feelepxyz's comment above about peer dependencies potentially being the root cause, CDK does use peer dependencies so that could be related.

For example, @aws-cdk/aws-certificatemanager has a peer dependency on @aws-cdk/core:
https://github.com/aws/aws-cdk/blob/v1.15.0/packages/%40aws-cdk/aws-certificatemanager/package.json#L84

@aws-cdk/core then has a peer dependency on @aws-cdk/cx-api:
https://github.com/aws/aws-cdk/blob/v1.15.0/packages/%40aws-cdk/core/package.json#L88

And the aws-cdk package (the only one that got upgraded) has a dependency on the same package @aws-cdk/cx-api.
https://github.com/aws/aws-cdk/blob/v1.15.0/packages/aws-cdk/package.json#L62

[ghost]

Any updates on this?

[feelepxyz]

We haven't made any progress on this. We're hoping to update to npm 7 and arborist which will give us a lot more powers to do peer dependency updates so will revisit once that lands.

[xlgmokha]

The version of npm that we use in core was bumped from 7.19.1 to 7.20.3 in #4110. It was released as part of dependabot-omnibus v0.158.0 in #4116. This version is trickling into production. We're continuing to monitor the progress.

If you have a public repo that you can point to that is having issues please share. Thank you for your patience.

[feelepxyz]

We haven't made any improvements to how dependabot-core handles peer dependencies so this is likely to be in a similar state. The npm 7 update might have improved which combinations raise peer dependency warnings so this might have inadvertently fixed some things.

[feelepxyz]

Looking at this a bit further, I think there are a few different issues that are bundled into "dependabot can't update x package".

If the dependency is top-level (defined in package.json):

• Dependabot can only update simple peer dependency requirements that point to the dependency being updated
  • For example, assuming you've installed react@15 and react-dom@15
    • When Dependabot attempts to update react from 15 to 16, it will see an error from npm: npm ERR! peer react@"^15.1.0" from [email protected]
    • This error is used to add react-dom to the react update, a version of react-dom is found that satisfies the new version of react
  • Dependabot will not update dependencies that have two-way peer dependency requirements defined, for example if react had a peer dependency on react-dom which in turn had a peer dependency on react, dependabot would bail out and not update either as it currently doesn't track if other updates have already updated a given dependency indirectly, so might end up with duplicate updates (we don't want to end up with 2 updates, one saying updating react and react-dom and the other saying updating react-dom and react)
  • If there are any nested peer requirements, for example if updating react-dom to v16 suddenly breaks several other dependencies, dependabot will probably bail out, but might in some cases give you a broken update 😕

If the dependency is transitive/sub-dependency:

• Dependabot can't update transitive dependencies beyond the latest allowed version by it's immediate parent dependency
  • Given the following dependency tree: [email protected] > espree@^4.0.0 > acorn@^5.6.0, given acorn is currently at 5.6.0, dependabot could only update acorn to 5.7.3 given the parent [email protected]. It's technically possible to update espree to 4.1.0 without updating it's parent eslint, which would allow us to install [email protected] but dependabot doesn't yet unlock/update transitive dependency parents.

We'd love to start tackling these issues and would also appreciate any help we can get here.

One way to help out would be to reduce problematic update scenarios to a minimal set of dependencies on a public repo with dependabot enabled. This would allow us to quickly write a test case, which is often the bulk of the effort when fixing these types of bugs.

[jeffwidman]

We've shipped a lot of improvements to peer dependencies / sub dependency bumping over the past few months: https://github.com/dependabot/dependabot-core/pulls?q=is%3Apr+peer+dependency+is%3Aclosed

Based on the issue descriptions ☝️, I'm fairly sure we've addressed a number of them... so I'm going to close as the concrete examples in this issue thread are more than two years old at this point. If you do hit this in the future, please file a new bug, and be sure to include both the conflict description as well as ideally a minimum reproducible example in a public repo that we can use for testing.