-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[python][pip-compile] Dependabot cannot resolve dependency version if the lockfile is used as constraints in other manifest files. #6550
Comments
Hei @nwalsh1995, thanks for reporting this. I just tried the latest command you present, but with the My guess is that user here is still on a I think we have two options here:
It's tricky, I don't think the solution is as simple as adding the |
@deivid-rodriguez can you try with these files instead? Without the PR I expect it to fail, and with the PR its expected to pass. Also, if possible can you tell me how you're testing the changes so that I can reproduce. Our files have extras stripped from Here are the files, you might have to play with version numbers to force an update of something out of date. Notice all of these files have been generated with requirements-shared.in
requirements-shared.txt
requirements-uses.in
requirements-uses.txt
|
Sorry, I misunderstood your issue, I thought your constraint files had extras and Dependabot was not being able to deal with them. You're saying that dependabot is adding the extras itself during them update, and then crashing because of that. Still I could not reproduce, I'm getting the same result, with or without your change. To test I pushed your manifests to a repo, downgrading I guess your issue happens when upgrading subdependencies? What's your configuration in |
@deivid-rodriguez can you try from this repo? It's closer to our actual usage and dependabot is failing there with the same logs as in the OP issue. https://github.com/nwalsh1995/depbot-example |
I see, you have extras in the constraints.in file not in the txt file. I still get the same error with your patch though. I think what pip is telling us is that it can't resolve if a constraints file (included through |
@deivid-rodriguez Another example which is closer to my initial report, this one doesn't contain extras in any of the files, and they are pulled in by dependabots run: https://github.com/nwalsh1995/depbot-example2 How are you running with the patch? It will be helpful for me to reproduce locally to see what's going on. |
Yes, but the files don't contain extras anywhere in the constraints file. Somehow extras are being generated.
I have no explanation of why the suggested PR doesn't work though, will need more info on how to debug that for me to help. |
Okay, I managed to repro the constraint problem locally on my docker instance ( I cannot repro the exception in depbot-example1 using |
From your PR branch, run |
For depbot-example its reproducible by running
Internally #6551 fixes the case in depot-example2 where someone uses There will have to be another issue/PR for handling manifest files with constraints which is reproducible in depbot-example. |
FYI I've created jazzband/pip-tools#1806 for |
Is there an existing issue for this?
Package ecosystem
pip/pip-compile
Package manager version
pip 22.3.1 pip-compile 6.12.1
Language version
3.7.5
Manifest location and content before the Dependabot update
No response
dependabot.yml content
No response
Updated dependency
No response
What you expected to see, versus what you actually saw
I expected dependabot to strip the extras out of this file since it is being used as a constraint to another file which dependabot will attempt to generate.
Instead dependabot does not strip the extras out of the produced file which means that the process fails with
Here is the code where dependabot runs
pip-compile
to resolve the version.dependabot-core/python/lib/dependabot/python/update_checker/pip_compile_version_resolver.rb
Lines 76 to 101 in 0a94b90
Given the manifest files below, notice how if we update a
.in
file that is being used as a-c
to another file, thenpip-compile
will fail to run on that file.Suggestion
There is a
pip-tools
discussion around making--strip-extras
the default behavior: jazzband/pip-tools#1613. Therefore, pass--strip-extras
by default in the twopip-compile
calls here:dependabot-core/python/lib/dependabot/python/update_checker/pip_compile_version_resolver.rb
Line 83 in 0a94b90
dependabot-core/python/lib/dependabot/python/update_checker/pip_compile_version_resolver.rb
Line 98 in 0a94b90
PR here #6551
Alternative to suggestion
dependabot supports parsing
--strip-extras
in the file_updater, it should support it in the version resolver as well:dependabot-core/python/lib/dependabot/python/file_updater/pip_compile_file_updater.rb
Line 442 in 0a94b90
Native package manager behavior
No response
Images of the diff or a link to the PR, issue, or logs
Smallest manifest that reproduces the issue
requirements-shared.in
requirements-shared.txt. This is what dependabot will generate because it doesn't pass
--strip-extras
. Notice requests[security]requirements-uses.in
Note: command below is what dependabot will effectively be running.
The text was updated successfully, but these errors were encountered: