python.defaultInterpreterPath doesn't always work (plus feedback/concerns about DeprecatePythonPath experiment)

[ex-nerd]

bug

After finding myself one of the 4% opted into the DeprecatePythonPath experiment, I've noticed some inconsistency in how the interpreter path is handled, and believe there is buggy behavior in how python.defaultInterpreterPath is implemented. The replication is as follows:

• Clear out your local storage so you start fresh
• Set python.defaultInterpreterPath to your preferred python path (e.g. a local pyenv installation of 3.7.2)
• Load up your project and notice that it detects the interpreter path properly
• Exit vscode
• Delete your python install and install a new one, simulating an upgrade (e.g. 3.7.4)
• Don't forget to update python.defaultInterpreterPath to point to the new interpreter path
• Reopen vscode and load up your project

Actual results:

vscode has now detected that the interpreter it had been using is no longer available, and is now in a state where there is no interpreter configured, and a user has to manually specify one.

Expected results:

vscode should detect that the cached state of the interpreter is incorrect (which it seems to do) but instead of "prompting" (it's hardly a "prompt") the user to select a new interpreter, it should fall back to the configured python.defaultInterpreterPath (after all, that's what a default is for).

Additionally:

• If you run this and choose not to fully delete the original python version, vscode will happily continue to use this old version instead of the one the project maintainers would prefer to enforce (see below for justification and use case).
• python envs installed in these subdirectories are not detected by vscode's picker and do not always show up with autocompletion so I had to manually type out (copy/paste) the entire path in order to use it (this seems to have improved after wiping my local storage cache directories, but is still annoying).

Other thoughts

I work on a team that uses direnv to enforce a specific environment for each of our projects. This includes maintaining different versions of python in the project directories based on where these will be deployed (e.g. App Engine has access to 3.8 but App Engine Flex only 3.7.2), and allowing us to rebuild these environments on the fly, test out different versions on different branches (e.g. when Flex gets 3.8 we'll make a branch to test that but still need to switch back and forth to work on "known stable" branches).

Since it seems likely from comments on #2125 that this experiment will become permanent, my initial solution to both of these was to add something the following to the shared settings.json files in each repository to ensure that everyone is opted in to this experiment and will default to the proper python instance:

{
  "python.experiments.optInto": ["DeprecatePythonPath - experiment"],
  "python.defaultInterpreterPath": "${workspaceFolder}/.direnv/python-3.7.2/bin/python3",
  // ...
}

However, due to the aforementioned bug, this means that many of our projects open up the "first" time with no interpreter set (seemingly conflicting with whatever value was cached before the experiment was activated), and that any attempts to upgrade or switch python interpreters for a specific project will mean the user has to manually specify the python interpreter every time he/she switches between branches with different python versions activated (and know which version to select, which means cracking open settings.json or our .envrc to see). This is not an improvement as far as usability goes.

So for now, I've opted my team out of this experiment (thanks for adding that option) until the usability issues have been resolved, and/or there are options that once again allow for project settings to control the python path:

{
  "python.experiments. optOutFrom": ["DeprecatePythonPath - experiment"],
  "python.pythonPath": "${workspaceFolder}/.direnv/python-3.7.2/bin/python3",
  // ...
}

[karrtikr]

@ex-nerd The steps to reproduce you mentioned didn't do it for me. For me the new interpreter was picked up correctly. What I did was,

• Selected a installed pyenv interpreter 3.5.2 using python.defaultInterpreterPath setting.
• Closed VSCode. Uninstalled the interpreter. Installed a new one 3.7.4
• Set python.defaultInterpreterPath to the new one
• Opened a python file to activate the extension again. The new one was picked up directly and no prompt was shown.

Note we do not cache interpreter setting values, and look up the settings everytime. We do cache the environments discovered, but not the setting.

Set python.defaultInterpreterPath to your preferred python path (e.g. a local pyenv installation of 3.7.2)

I suspect you have this interpreter set in workspace settings (which are now stored in VSCode internal storage) instead of user setting python.defaultInterpreterPath, which is why it is being picked up. Please see wiki for more details on how interpreters are stored in the experiment.

Try to clear the value stored in workspace settings using Python: Clear Workspace Interpreter Setting command, and then your user setting python.defaultInterpreterPath value should be picked up correctly.

[ex-nerd]

@karrtikr If by "in workspace settings" you mean that the default interpreter path and all other settings are stored in .vscode/settings.json in the folder in my git repository, yes. That's the whole point of configuring a separate interpreter for each of our projects. vscode is usually invoked by cd into the same directory as the source and running code . (to pick up several environment variables that only exist within that directory)

If you're unable to reproduce this by using global settings, then it sounds like this is definitely a bug if vscode handles the workspace settings differently.

[karrtikr]

No I don't mean that at all. Please have a look at the wiki I linked earlier to see how settings are stored. And try the steps I mentioned.

User and workspace settings are two entirely different things.

[ex-nerd]

@karrtikr As far as I can tell, the wiki is saying that my situation is an intentional choice to break projects that use workspace-specific interpreters controlled by the workspace itself:

A new user setting python.defaultInterpreterPath is introduced which is meant as a replacement for python.pythonPath in user scope, but not in workspace scope.

I'm trying to use python.defaultInterpreterPath in a workspace scope. The whole point of filing this bug is users shouldn't have to go out of their way to manually find/select an interpreter path if the workspace settings know what it should be (and it would be an error if that interpreter doesn't exist ... falling back to some other copy of python could result in bad code being added).

[karrtikr]

I'm trying to use python.defaultInterpreterPath in a workspace scope.

You cannot use a user setting in workspace scope.

Like the wiki mentions, if you want to configure workspace-specific interpreters, use the Python: Select Interpreter command instead of python.defaultInterpreterPath.

Please try the solution I mentioned earlier with clearing workspace settings and then get back to me.

[ex-nerd]

@karrtikr Yes... Granted, part of this is confusion around the experiment info page I was sent to when vscode notified me that I'd been opted in (which to my recollection didn't mention that python.defaultInterpreterPath was not allowed in workspace settings, simply that it replaced python.pythonPath with slightly different functionality).

There also seems to be something different between using a pyenv environment from its main location in ~/.pyenv/ and the actual active one installed as a virtual environment for each of my projects.

But you keep answering me as if I'm not specifically reporting it as a bug that I can't control the python interpreter from my workspace settings like I can if I opt out of this experiment and revert to previous behavior. This new experimental behavior previous convention and forces users to manually type in a path they may not know every time the project changes its interpreter settings (because for some reason it doesn't show up in the picker .. and I can't even use something like python.defaultInterpreterPath in the workspace config to ensure that it does show up in the picker). The experiment asked for feedback and told me to come here to provide it. So I'm providing feedback: the current implementation breaks legitimate use cases where the project maintainers (e.g. my employer and several previous ones) use a .vscode/settings.json file committed into the code base to ensure that all developers use the exact same version of python, which includes automatically switching them to new/different versions as they switch between branches. Storing manually-specified values outside of the project workspace is fine ... but it is a mistake to remove the ability for the workspace config to specify what its defaults should be.

[karrtikr]

Yes... Granted, part of this is confusion around the experiment info page I was sent to when vscode notified me

Yes, apologies for the confusion. The previous wiki wasn't clear enough about the details, I recently mended it.

But you keep answering me as if I'm not specifically reporting it as a bug

I wanted to first clear up the first part where you mentioned you "believe there is buggy behavior in how python.defaultInterpreterPath is implemented", and then get to your other thoughts section (which is a very valuable feedback, thanks for that).

We realize that the new experiment would take away the ability to share interpreter paths in workspace settings, which is why I wasn't talking about it like it's a bug because the behavior you see is expected from dev-perspective. If interested, you can have a look at the few issues which motivated this solution.

We have marked this issue as "needs decision" to make sure we have a conversation about your thoughts, after which we'll get back to you.

[ex-nerd]

Thanks, and that's fair. The original wording I used was based on the unclear wording in the wiki about the use of python.defaultInterpreterPath.

I think much of this could be solved by making python.defaultInterpreterPath a setting allowed by the workspace settings. As a "default" it would only come into play if the user does not have an interpreter set up, and it allows projects to provide a suggested/preferred interpreter to use (which is really important when so many python projects have both minimum and maximum python version dependencies due to differences in python 2.7, 3.5, 3.7, 3.8, and virtual envs are so prevalent to limit interactions between third-party packages).

Though a better idea might be to add a different workspace-specific setting python.recommendedInterpreterPath, allowing for the user to be prompted when their configured path is different ... sort of like how a project can have recommended extensions and the user is notified if any of those are not installed (and those who want to can dismiss the recommendation in such a way that it never reappears). My biggest concern with this is the user experience of the whole thing:

1. The project's preferred python path didn't show up in the menu to select a new interpreter so I had to manually track it down in the terminal and paste in the entire path
   • This part may simply be a caching issue, since restarting vscode in between re/creating the python environments did seem to let me type in each directory name to traverse my way to the proper python version (I only tested "restart in between" once so something else may have triggered the "fixed" behavior).
2. There is no way to notify the user that their currently-configured path is "incorrect" (for a specific interpreter preferred/needed by the project) or at least "incompatible" (for projects that are only compatible with certain python versions).
   • Basically, it's unhelpful to tell the user "hey, you don't have python configured for this workspace ... please pick your interpreter" while also preventing the project from providing even a suggestion to the user which one they should use.
   • Simply allowing the user to try to use an incompatible interpreter would trigger all kinds of unnecessary warnings in the IDE and potentially waste the user's time tracking down "problems with the code" when it simply's an incorrect environment. In fact, this is my primary concern, since one of the main reasons we use per-project project-managed python envs is specifically because so much time has been wasted helping other developers track down issues that turned out to simply be related to a misconfigured or out of date environment.

[xabinapal]

Today I just found myself in the exact same scenario as @ex-nerd reports. Our team also uses direnv and sets a different Python environment for each project, which is always relative to the workspace directory. We have always used the pythonPath setting in our workspace directories (.vscode/settings.json), but now this does not work anymore.

Yes, we can make our developers to manually select an environment in each one of their projects, but this is not intuitive (sometimes the required environment is not shown even though it's in the $PATH and, more important, this requires manual action!), is error-prone (many of our developers don't know the "internals" of our direnv structure and can select the wrong interpreter), and does not consider future version upgrades (if after some development we require a different Python version, we must tell our developers and ensure that they change their settings).

So, yes, we understand the need to allow a user setting for all the scenarios where Python paths are not shared, but I don't think that disabling the existing behaviour is a good idea as it breaks compatibility with some use cases and, even worse, it forces teams to waste work time trying to discover how this must be done now.

This should be as easy as testing the pythonPath parameter (or defaultInterpreterPath, recommendedInterpreterPath, overrideInterpreterPath or any other recognizable name) and if not available or not set, use the internal user setting.

[arterrey]

Please don't implement this feature. Or at least allow for defaultInterpreterPath to be set at the workspace level, many projects I've worked on manage Python interpreters internally to the project - for the exact same reasons stated as the motivation of this experiment "teams that don’t use the same OS, or simply work with interpreters that don’t share the same path" - but more then that often the interpreter is managed in the project precisely to isolate the project from system python. It is a major inconvenience to need to explicitly set this every time - and looks to be error prone as it isn't clear that it is not set or that it needs to be set differently.

[garthwilliamson]

According to https://github.com/microsoft/vscode-python/wiki/AB-Experiments this is now available in workspace scope. Is that correct?

[karrtikr]

Yep, this can now be closed.