Provide a programmatic API for pip

[freakboy3742]

What's the problem this feature will solve?

Most users of pip will only ever need the command line interface. However, there is a small subset of use cases where having programmatic access to pip's functionality can be helpful.

For example:

• Meta-installation tools, like briefcase, use pip to perform installations of dependencies for the application being packaged.

• Development environments that provide a Python environment for behavioural scripting can invoke pip to install dependencies being used in code.

Describe the solution you'd like

A Python API equivalent to invoking pip from the command line.

This could be as simple as formally exposing the same main method that is used by pip's mainline:

import pip
pip.main('install', '-U', 'package1', 'package2')

Or, exposing specific functionality with individual functions:

import pip
pip.install('package1', 'package2', upgrade=True)

Alternative Solutions

It's possible to use subprocess to invoke pip as a shell command. However, this can be difficult to implement in a reliable cross-platform fashion. Invoking pip via shell command also makes error handling more complex, as all pip-level errors must be proxied through operating system-level error status codes.

Additional context

This is a feature that previously existed, and has been removed. Prior to pip 10, pip.main could be imported and invoked. Pip 10 moved the main() method into the _internal namespace, making it clear the API wasn't to be used; however, if you ignored that guidance, it could still be imported and used. Pip 18 appears to have made additional changes that rendered this workaround non-functional.

The absence/removal of pip.main has been reported (many times) as a bug; #5599 gathers a redux of those reports. However, those reports and diagnoses all appear to be related to problems that are observed when pip has been installed or upgraded badly. The reference to pip.main in stack traces appears to be incidental.

This report is about a specific piece of programmatic functionality that was effectively removed in pip 10, but still has utility for some users (albeit an admittedly small subset).

[pfmoore]

There was never a supported API for pip. People could use the unsupported API, but that doesn't mean it's a good thing to do. We had a number of "bug reports" for such uses that involved things like:

• Things don't work right if I'm using threads in my program (even when not using them to call pip)
• Pip's use of logging interferes with how I want to use it
• Stuff pip does interferes with global state in a way that affects my code

See the tracker for further details, but the response was essentially always "don't do that, then".

You say "Pip 18 appears to have made additional changes that rendered this workaround non-functional" - that's precisely the sort of thing that explains why we don't support calling pip from your program. We do make changes that break internal APIs, and that's why we don't support using those APIs.

Requests along the lines of "please add a supported programming API for pip" have been made before, and you can find the discussions and reasons as to why we haven't done so if you search the tracker (basically "we don't have the manpower, and it's a lot more work than you're assuming"). But claiming that pip used to have this functionality and it was removed is just flat-out wrong. The functionality was either never there (because we never supported it) or not removed (we just made some changes to unsupported APIs, deal with it) depending on your perspective.

There's not really anything new to add here.

[pradyunsg]

This is a duplicate of #3121.

There's a few other issues in the past where this has been discussed - they're tagged with the public API tag.

[freakboy3742]

My choice of phrasing was imprecise - I didn't intend to imply that the pre-pip 10 API was an officially supported feature; only that an API existed, and wasn't as clearly marked as "internal" in the way that was in pip 10.

I did try to search for other similar requests, but was unable to find any - possibly obscured by the errors that arise as a result of bad installs and upgrades to pip that yield very similar errors.

Regardless, it sounds like the answer is that the API was very intentionally hidden, and there's no particular desire to bring it back as an official API capacity, so I'll need to explore other options.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.