Changes to logic for selection of best (default) version of Python interpreter

[DonJayamanne]

Related to #3326

Today we use python as the default version of Python for the extension.

Logic for selection of best (default) version of Python interpreter:

• Pick interpreter with highest version and bitness

[DonJayamanne]

Requirement

• The extension needs to chose the best available Python Interpereter

Problems

• How to quickly identify best available Python Interpereter
• We store selected python in python.pythonPath in settings.json
• Scenario 1:
  • User has Python 3.6 installed (in path)
  • Open workspace, and extension uses Python 3.6.
  • User installs pylint, flake8, etc in the above global environment.
  • Tomorrow user installs Python 3.7 and opens the above workspace
  • They'll need to install all the python dependencies again
  • Yes virtual envs are a solution, but not necessarily the right one.
• Scenario 2:
  • User install VSC for first time on Windows
  • User opens a new workspace
  • Extension is running in the background attempting to determine the best available Python version
  • This can take > 30 seconds on Windows with Conda.
  • Extension is not usable a the best available version of Python hasn't been identified

Terminology

• Store in workspace cache
  • Rather than storing information in settings.json file, we can store in a cache thats specific to a workspace.
  • This way, we do not write to settings.json, and the information stored is specific to a workspace.

Solutions:

• How to quickly identify best available Python Interpereter

  • On Mac and Linux look for python3.7, python3, etc in current path.
  • On Windows look for py -3.7, etc in current path
  • If all of the above fail, then we use windows registry, conda, etc...
  • Which is available, pick that.
  • What ever algorithm we use, this MUST be simple and very quick
  • Once extension loads, we can iterate through all known interpreters an pick best. I.e. worst case scenario, first time we might not pick the best interpreter.
  • Again, if by the time the best available is determined, and we have a list of ALL interpreters, then we just go and pick best from ALL interpreters.
  • Lets discuss this item seprately

• A. First time extension is installed and loaded:

  • A1: Look for local virtual envs (or pipenv), if found use that & store in workspace cache.
  • A2: If previous step was unsuccessful, then pick best available python interpreter (algorithm outline separately - see above) and store that in workspace cache as well as global cache.
  • A3: In the mean time, look for all interpreters (in the background), if we find a version of Python that's better than that was identified and stored earlier (A1/A2), then store this in global cache for next time..

• B. Open new workspace (not first time loading extension)

  • B1: Look for local virtual envs (or pipenv), if found use that & store in workspace cache usual place as done today, i.e. settings.json.
  • B2: Else, get python interpreter stored in global cache (identified in previous A2/A3.
  • B3: In the mean time, look for all interpreters (in the background), if we find a version of Python that's better than that was identified and stored earlier, then store this in global cache for next time..

• C. Open existing workspace (not first time loading extension)

  • C1: Check if we python defined in python.pythonPath in settings.json, if yes, then use that.
  • C2: Check if we have stored details of python interpreter in stored in workspace global cache. If yes, then use that.
  • C3: Look for local virtual envs (or pipenv), if found use that & store in workspace cache do what we do today, i.e. store in usual place - settings.json.
  • C4: In the mean time, look for all interpreters (in the background), if we find a version of Python that's better than that was identified and stored earlier, then store this in global cache for next time..

[brettcannon]

One thing we will need to help with debugging reported issues is a way for people to get to the path to the selected Python interpreter. Maybe a command to write it out to settings.json?

[DonJayamanne]

Sounds like we need to create a command that'll spit out debug logs

• Python path
• Versions
• Settings
• Launch config

Get all info and dump into clipboard or open an editor with all of that in there for user to copy and paste into the GitHub issue.

[ericsnowcurrently]

This mostly worked for me on master. Instead of picking "python" as it used to, it picked Python 3.7.0 64-bit ('.env': virtualenv). The following were available:

• Python 2.7.15 64-bit
• Python 3.5.2 64-bit
• Python 3.6.4 64-bit ('.miniconda3': conda)
• Python 3.6.7 64-bit
• Python 3.7.0 64-bit ('.env': virtualenv)
• Python 3.7.0 64-bit ('test env': conda)
• Python 3.7.0 64-bit ('3.7.0': pyenv)
• Python 3.7.1 64-bit ('base': conda)
• Python 3.7.1 64-bit

So it's not clear why it did not pick 3.7.1, nor why it preferred the virtualenv over conda or pyenv (for 3.7.0). I would have expected it to pick 3.7.1. In the face of a tie (as with 3.7.0) I would have expected it to ask me.

All that said, picking 3.7.0 instead of 2.7 is an improvement. :)

[brettcannon]

@ericsnowcurrently the order preference is venv, virtualenv, pipenv, global, conda; IOW this worked the way it's designed. The thinking is the first three options were on purpose and are directly tied to the workspace folder, and conda is last because (a) it isn't directly tied to a directory and (b) it's slow to detect 😁 .