Dependabot fails to update same dependency in same pull request

[glensc]

Is there an existing issue for this?

• I have searched the existing issues

Package ecosystem

pip, pipenv

Package manager version

No response

Language version

No response

Manifest location and content before the Dependabot update

No response

dependabot.yml content

No response

Updated dependency

No response

What you expected to see, versus what you actually saw

Dependabot created two PR's for same dependency update:

• https://github.com/Taxel/PlexTraktSync/pull/1209/files - updates requirements.txt
• https://github.com/Taxel/PlexTraktSync/pull/1216/files - updates Pipfile.lock, and python_version < '4' to python_version < '4.0' changes already reported as Dependabot flip-flop with python_full_version/python_version #6091

it used to manage this nicely and update both files in same pull request. sometimes it fails, i have not yet determined the pattern.

Native package manager behavior

No response

Images of the diff or a link to the PR, issue, or logs

No response

Smallest manifest that reproduces the issue

No response

[deivid-rodriguez]

Looking at your dependency files, I think there's two issues here:

• In presence of Pipfile and Pipfile.lock file, a requirements.txt file should be considered just as another lock file, and don't get updated directly, but just be updated to follow Pipfile.lock changes. So no PRs should be proposed for you unless you configure allow: [{ dependency-type: "indirect" }] in your dependabot.yml file.

• If allow: [{ dependency-type: "indirect" }] was configured, you should indeed get just one PR with both updates included, not two.

[glensc]

@deivid-rodriguez so, if I got you correctly, do I need to do this?:

• Dependabot: Set dependency-type: "all" Taxel/PlexTraktSync#1218

[deivid-rodriguez]

I think that's how it should work, yes. By default you should only get updates for dependencies explicitly defined in your Pipfile.

But actually, you probably want dependency-type: all since otherwise you're opting out of direct dependencies.

But in any case, I don't think it's going to make a difference right now due to this bug, but feel free to leave feedback on whether the change had any effect on your repo.

[glensc]

i'll go with all then. thanks

[glensc]

I've merged that, but now it still updated only one file, requirements.txt this time:

• Bump urllib3 from 1.26.12 to 1.26.13 Taxel/PlexTraktSync#1226

[glensc]

oh, wow, it created the second PR already so soon:

• Bump urllib3 from 1.26.12 to 1.26.13 Taxel/PlexTraktSync#1232

[pavera]

I started looking at this in the debugger and it looks like it's possibly an issue in the resolver_type method which basically picks which package manager we think is being used. In my tests it keeps picking plain pip based on the fact that it is finding a possible update in requirements.txt before finding the Pipfile update.

I'll keep pulling on this thread and see if I can come up with a solution.

[jeffwidman]

This is not the first time we've run into problems with trying to guess which package manager is being used within an ecosystem. From a product perspective IMO we should reconsider if it might be more straightforward for us to ask users to specify the specific manager for any ecosystem with more than one possible package manager. We probably could pick a default for every ecosystem. This would also pet users specify more than one package manager if they wanted to track updates multiple paths. Anyway, a discussion beyond the scope of this particular issue but wanted to mention as a recurring thread I've seen.

(the other recurring thread I've seen is the confusion resulting from python treating requirements.txt as both a manifest and a lockfile. Life would be simpler if they had an officially-blessed lockfile that all package managers generate towards... I wonder if anyone has ever submitted a PEP for that?)

[glensc]

Another example:

• Bump prompt-toolkit from 3.0.33 to 3.0.36 Taxel/PlexTraktSync#1259
• commit: Taxel/PlexTraktSync@e39cddf (#1259)
• requirements.txt bumped 3.0.33 to 3.0.36
• Pipfile.lock - stayed at 3.0.33: https://github.com/Taxel/PlexTraktSync/blob/e39cddfc2799d06e88425dbd644881f7c53452da/Pipfile.lock#L136

[pavera]

Yeah, our logic here is a little fuzzy and doesn't really work for transitive dependencies. Basically since prompt-toolkit is a transitive dependency, we don't see that it needs to be updated in the pipenv environment because it is not in the Pipfile. When we're detecting which package manager we're operating as we only consider the "manifest" files currently, which we classify requirements.txt as a manifest file. Since the dependency doesn't occur in Pipfile we choose pip instead of pipenv and perform the update only on requirements.txt instead of choosing pipenv which would then update both.

I'm attempting to improve this logic to prefer pipenv over pip in the presence of a Pipfile even if the dependency doesn't occur in said Pipfile. Hopefully will have a PR in the next day or so. Basically the current assumption is if there is a Pipfile and a requirements.txt the requirements.txt is probably managed directly with pip and not generated from the Pipfile. My approach is going to be to change this assumption to be if there are both files in the repository, assume the requirements.txt is generated from pipenv until shown otherwise.

[glensc]

ok, so for me workaround is to just add all dependencies to both files. the requrements.txt is just a dump of pipfile.lock anyway in that project.

EDIT: Added:

pipenv install prompt-toolkit==3.0.33

• Taxel/PlexTraktSync@5279a85

[glensc]

And now there's another problem: It updates other unrelated dependencies as well:

• https://github.com/Taxel/PlexTraktSync/pull/1266/files

The PR is about "Bump certifi from 2022.9.24 to 2022.12.7", which it happens to update Pipfile, Pipfile.lock, requirements.txt. so far so good.

but it also updates prompt-toolkit==3.0.36; python_full_version >= '3.6.2', which there was prior PR already:

• Bump prompt-toolkit from 3.0.33 to 3.0.36 Taxel/PlexTraktSync#1259

[pavera]

I'm not seeing the prompt-toolkit update in that PR? In requirements.txt it just seems to be removing the python_full_version from the prompt-toolkit line not changing the version? So it seems this is just another example of your issue #6091? Or am I missing another update to prompt-toolkit in that PR?

[glensc]

it was in the previous push:

@dependabot dependabot bot force-pushed the dependabot/pip/certifi-2022.12.7 branch from 36243f3 to 8e5462c 1 hour ago

So see this commit:

• Taxel/PlexTraktSync@36243f3

[glensc]

How do I add items with markers to Pipfile?

i.e requirements has:

attrs==22.1.0; python_version >= '3.5'

I'm trying to work around that dependabot updates Pipenv files as well files in this PR:

• Taxel/PlexTraktSync@022256a (#1301)

[glensc]

Since there was already marker in Pipfile.lock:

• https://github.com/Taxel/PlexTraktSync/blob/3f5fa8b8be02d4613304c233af138ba58f43ac6f/Pipfile.lock#L27-L34

I only added line to Pipfile without markers

• Taxel/PlexTraktSync@f3a5e2d

[abdulapopoola]

Closing out as stale

[glensc]

Some kind of problem still there. Need to add all packages to Pipenv to avoid some buggy behavior:

• Pipenv: Track typing-extensions package for dependabot Taxel/PlexTraktSync#1983