Environment Id is not static (it changes) when installing Python into an empty Conda environment

[DonJayamanne]

• Create a conda enviornment without Python and the id is the sys prefix of the conda enviornment
• Next install Python into this conda environment
• Now, reload VS Code, and the id for this environment is no longer the same.

@karrtikr I'd expect the Id to be the same everytime and not change.
Can we have the env id to be always sysprefix or the like, somethign that will never change for the same environment.

[DonJayamanne]

This is crucial for Jupyter extension as we create Kernels (somethign you see the Kernel quick pick) & we have an Id associated with each controller.
VS Code remembers this id.

• Hence assume you create a conda env without Python
• Now select this kernel
• & then later you instlal Python into this
• Now reload VS code,
• The controller no longer exists as the Id has now changed, hence Jupyter extnesion is no longer able to preserve the users selection

@rebornix /cc

[karrtikr]

Currently installing Python into the environment is considered the same as deleting the environment and creating an entirely new one, so the ID doesn't change per se as it's a new environment.

The selection can still be maintained however, resolveEnvironment returns the same environment if you pass the path to environment folder or the path to the executable. So even if ID changes the same environment is returned.

[karrtikr]

Can we have the env id to be always sysprefix or the like

Let me know if testing above works, otherwise I'll see if we can simply do this. Due to historical reasons this wasn't possible earlier.

[DonJayamanne]

Let me know if testing above works, otherwise I'll see if we can simply do this. Due to historical reasons this wasn't possible earlier.

Can't you instead set the Id as the Uri of the python executable?
Else at our end we need to track this and keep our own Ids.
I.e. the ID returned by Python extension will not be useful to us and we'll need to invent our own Id which will be based on the executable (in the case of conda without Python, the Id would be the python executable if such a file were to exist, this way in jupyter when Python is installed the id is still the same.
& we can use another flag to determine whether python exists or not, e.g. based on sysPrefix or another flag all together.

[karrtikr]

Discussed offline, marking this as something that can be covered in debt week.

[karrtikr]

We discussed offline that Jupyter was passing id to resolveEnvironment, which I think was the reason why Jupyter is not able to maintain the selection here: #20176 (comment). microsoft/vscode-jupyter#11949 should fix it but this issue is still a nice to have if the ID doesn't change.

[DonJayamanne]

@karrtikr can we change this to a feature request or bug, I do not think this is a debt item,

If Python extension were to use the same API internally, the Python extension too would run into this exact same issue. However I can see that the Python extension is genearting its own id and using the Python executable path as the Id and the executable doesn't exist. But assumed to exist. I.e. its kind of a work around.

The Jupyter extension too could work around this by generating a custom ID instead of the ID that Python provides, but that would be a work around for a bug in Python extension, hence the request for this to be treated as a bug and not a debt item.

The selection can still be maintained however, resolveEnvironment returns the same environment if you pass the path to environment folder or the path to the executable. So even if ID changes the same environment is returned.

The selection is tied directly to an environment and if that ID changes, it is not the same.

So even if ID changes the same environment is returned.

From an API perspective this is not the same, its impossible to treat them as the same if the Ids are different. Else I fail to understand the whole point of having an ID (saying its the same yet having different IDs makes no senes)

[karrtikr]

Changing labels, I only marked it as debt so I can remember to cover it in some debt week.

However I can see that the Python extension is genearting its own id and using the Python executable path as the Id and the executable doesn't exist. But assumed to exist

Btw where is that? We use the same id in the external surface and internally AFAIK.

[karrtikr]

I understand it's not ideal, I'm just wondering why is it critical for Jupyter when Python extension seems to work fine with it.

[DonJayamanne]

Python extension seems to work fine with it.

I've mentioned this here #20176 (comment)

Python extension is not using the Id to uniquely identifiey Python environments. If the Python extension too used the same API internally Python extension too would run into this exact same issue.
From the code I san see that you use the Python executable path to identify Python environments. I.e. when user selects a Python env that is an empty conda env, then the python exucutable is used and stored. & subsequently python extension usees that to get the interpreter details and figgure out whether pytyhon is installed or not and then isntalls it

I suggest chaning the code to use the Python env ID instead of the executable, that should make the issue more obvious in the Python extension. At this point the selected interpreter would change for Python extension as well.

As mentioned earlier Python internally does not use the same ID, its using a work around and generating its own Id , and Jupyter too would add such a work around.

[karrtikr]

when user selects a Python env that is an empty conda env, then the python exucutable is used and stored.

Where is that? We use .pythonPath, which can be either environment path or the python executable.

[DonJayamanne]

Ahh my mistake, yes you are using the env path.
Its in src/client/interpreter/interpreterService.ts
This highlights my point, Python extension is not using the ID to uniquely identify Python environments, hence Python does not run into this issue (i.e. internally the same API is not used)

[karrtikr]

It is the same as ID, except for the case. I don't think Jupyter is using the API as I'm envisioning it, what're you passing to resolveEnvironment for environments without python, and what is the issue you're seeing?

[DonJayamanne]

I've created an env
and is is what I get from Python API

{"internal":{"path":"/Users/donjayamanne/miniconda3/envs/tes05","id":"/Users/donjayamanne/miniconda3/envs/tes05","executable":{"bitness":"Unknown","sysPrefix":""},"environment":{"type":"Conda","name":"tes05","folderUri":{"$mid":1,"path":"/Users/donjayamanne/miniconda3/envs/tes05","scheme":"file"}},"tools":["Conda"]},"id":"/Users/donjayamanne/miniconda3/envs/tes05"} 

After installing Python then the output from Python API is different

 {"internal":{"path":"/Users/donjayamanne/miniconda3/envs/tes05/bin/python","id":"/Users/donjayamanne/miniconda3/envs/tes05/bin/python","executable":{"uri":{"$mid":1,"path":"/Users/donjayamanne/miniconda3/envs/tes05/bin/python","scheme":"file"},"bitness":"64-bit","sysPrefix":"/Users/donjayamanne/miniconda3/envs/tes05"},"environment":{"type":"Conda","name":"tes05","folderUri":{"$mid":1,"path":"/Users/donjayamanne/miniconda3/envs/tes05","scheme":"file"}},"version":{"major":3,"minor":10,"micro":8,"release":{"level":"final","serial":0},"sysVersion":"3.10.8 (main, Nov 24 2022, 08:08:27) [Clang 14.0.6 ]"},"tools":["Conda"]},"id":"/Users/donjayamanne/miniconda3/envs/tes05/bin/python"} 

It is the same as ID, except for the case

As you can see the IDs are different.

[DonJayamanne]

Here are more logs
We pass in the environment for resolving
Before (fully resolved env)

{"internal":{"path":"/Users/donjayamanne/miniconda3/envs/tes06","id":"/Users/donjayamanne/miniconda3/envs/tes06","executable":{"bitness":"Unknown"},"environment":{"type":"Conda","name":"tes06","folderUri":{"$mid":1,"path":"/Users/donjayamanne/miniconda3/envs/tes06","scheme":"file"}},"tools":["Conda"]},"id":"/Users/donjayamanne/miniconda3/envs/tes06"} 

Before (env returned in known property)

{"internal":{"path":"/Users/donjayamanne/miniconda3/envs/tes06","id":"/Users/donjayamanne/miniconda3/envs/tes06","executable":{"bitness":"Unknown"},"environment":{"type":"Conda","name":"tes06","folderUri":{"$mid":1,"path":"/Users/donjayamanne/miniconda3/envs/tes06","scheme":"file"}},"tools":["Conda"]},"id":"/Users/donjayamanne/miniconda3/envs/tes06"} 

After (fully resolved env)

 {"internal":{"path":"/Users/donjayamanne/miniconda3/envs/tes06/bin/python","id":"/Users/donjayamanne/miniconda3/envs/tes06/bin/python","executable":{"uri":{"$mid":1,"path":"/Users/donjayamanne/miniconda3/envs/tes06/bin/python","scheme":"file"},"bitness":"64-bit","sysPrefix":"/Users/donjayamanne/miniconda3/envs/tes06"},"environment":{"type":"Conda","name":"tes06","folderUri":{"$mid":1,"path":"/Users/donjayamanne/miniconda3/envs/tes06","scheme":"file"}},"version":{"major":3,"minor":10,"micro":8,"release":{"level":"final","serial":0},"sysVersion":"3.10.8 (main, Nov 24 2022, 08:08:27) [Clang 14.0.6 ]"},"tools":["Conda"]},"id":"/Users/donjayamanne/miniconda3/envs/tes06/bin/python"} 

After (env returned in known property)

{"internal":{"path":"/Users/donjayamanne/miniconda3/envs/tes06/bin/python","id":"/Users/donjayamanne/miniconda3/envs/tes06/bin/python","executable":{"uri":{"$mid":1,"path":"/Users/donjayamanne/miniconda3/envs/tes06/bin/python","scheme":"file"},"bitness":"64-bit","sysPrefix":"/Users/donjayamanne/miniconda3/envs/tes06"},"environment":{"type":"Conda","name":"tes06","folderUri":{"$mid":1,"path":"/Users/donjayamanne/miniconda3/envs/tes06","scheme":"file"}},"version":{"major":3,"minor":10,"micro":8,"release":{"level":"final","serial":0},"sysVersion":"3.10.8 (main, Nov 24 2022, 08:08:27) [Clang 14.0.6 ]"},"tools":["Conda"]},"id":"/Users/donjayamanne/miniconda3/envs/tes06/bin/python"}

It is the same as ID, except for the case. I don't think Jupyter is using the API as I'm envisioning it, what're you passing to resolveEnvironment for environments without python, and what is the issue you're seeing?

I'm confused here, I thought after our discussion you already acknowledged this was expected.
see here #20176 (comment)

[karrtikr]

@DonJayamanne Can you verify this for Jupyter?