Fixing PR failures if pypi.org unavailable

[pavera]

Context

There are many ways to specify an alternate pypi index in Python depending on the package manager, for the most part we attempt to parse these settings from pyproject.toml, pip.conf, and requirements.txt settings. If the project specifies an alternate index url we shouldn't call the public pypi.org index.

There are a number of calls we make to pypi.org to retrieve data that is only available there (primarily the <package name>/json metadata endpoint). These calls currently cause PR creation to fail if pypi.org is unavailable or otherwise blocked/firewalled. We should be able to generate a PR given a private index even if pypi.org is unavailable.

Approach

There are 3 classes of issues I found in attempting to run updates without pypi.org access:

1. Failures to get metadata from pypi.org resulted in failed PR creation
2. Our python version detection was failing to detect already installed python versions resulting in extra attempts to install python and pip from pypi.org
3. pip-compile is not properly injecting an alternate index url provided in a requirements.txt file

I've added a couple rescues to catch timeouts so failed attempts to reach pypi.org do not crash pr creation. These could probably be expanded further to include other network errors (SocketError maybe?)

Calls to pyenv versions were always failing to find installed python for pipenv and pip-compile as they were expecting a newline which is no longer in the output. This "fix" only resolves the issue if the repo is using a version of python we have pre-installed.

I have not attempted to address point 3 above yet, I added a partial solution of exposing the main_url from IndexFinder via a new public method on IndexFinder and this worked, however it would require a larger change to properly set this in context with replaces_base and I feel there might be better options to explore around parsing the exact command pip-compile recommends from the requirements.txt file. Since pip-compile supports multiple source files (requirements.in, setup.py, and pyproject.toml) we should probably take a more holistic approach here.

[landongrindheim]

Is the preceding space important here? If so, what do you think about using "\ #{? I'm able to notice it here because of diff highlighting, but I'd be much less likely to notice it when I need to.

[pavera]

Yes the space is important, I've applied the suggested change.