Run cell does not use virtualenv

[jayanthkoushik]

Environment data

• VS Code version: 1.28.2
• Extension version (available under the Extensions sidebar): 2018.10.1
• OS and version: macOS High Sierra 10.13.6
• Python version (& distribution if applicable, e.g. Anaconda): 3.7.0
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): pipenv
• Relevant/affected Python packages and their versions: all

Actual behavior

IPython kernels spawned on running cells does not use virtualenv, causing imports to fail.

Expected behavior

IPython from virtualenv should be used

Steps to reproduce:

1. Create virtualenv
2. Set python.pythonPath to virtualenv in .vscode/settings.json
3. Install ipython, jupyter, and some other package (say matplotlib) in virtualenv
4. Create a python file with just import matplotlib
5. Add #%% and run cell
6. Notebook preview opens and ImportError is thrown

Logs

Running get_ipython().run_line_magic("matplotlib", "inline") reveals that the wrong IPython is used:

get_ipython().run_line_magic("matplotlib", "inline")
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
 in 
      1 # %%
----> 2 get_ipython().run_line_magic("matplotlib", "inline")
      3 
      4 print("hello, world")

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
   2283                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
   2284             with self.builtin_trap:
-> 2285                 result = fn(*args,**kwargs)
   2286             return result
   2287 

 in matplotlib(self, line)

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/IPython/core/magic.py in (f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188 
    189         if callable(arg):

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/IPython/core/magics/pylab.py in matplotlib(self, line)
     97             print("Available matplotlib backends: %s" % backends_list)
     98         else:
---> 99             gui, backend = self.shell.enable_matplotlib(args.gui)
    100             self._show_matplotlib_backend(args.gui, backend)
    101 

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/IPython/core/interactiveshell.py in enable_matplotlib(self, gui)
   3339         """
   3340         from IPython.core import pylabtools as pt
-> 3341         gui, backend = pt.find_gui_and_backend(gui, self.pylab_gui_select)
   3342 
   3343         if gui != 'inline':

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/IPython/core/pylabtools.py in find_gui_and_backend(gui, gui_select)
    274     """
    275 
--> 276     import matplotlib
    277 
    278     if gui and gui != 'auto':

ModuleNotFoundError: No module named 'matplotlib'

Output from Console under the Developer Tools panel.

console.ts:136 [Extension Host] Python Extension: [I 21:00:15.098 NotebookApp] Serving notebooks from local directory: /var/folders/7_/vlgt1szj3gdcw4x8v587kg2m0000gn/T

console.ts:136 [Extension Host] Python Extension: [I 21:00:15.099 NotebookApp] The Jupyter Notebook is running at:
[I 21:00:15.099 NotebookApp] http://localhost:8888/?token=a9dcd3804f44df60a6cb2a29cab9442918c8c758d7f849a0
[I 21:00:15.099 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

console.ts:136 [Extension Host] Python Extension: [C 21:00:15.106 NotebookApp] 
    
    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=a9dcd3804f44df60a6cb2a29cab9442918c8c758d7f849a0

console.ts:136 [Extension Host] Python Extension: [I 21:00:15.287 NotebookApp] Kernel started: e4c82ade-2da6-43ae-81c8-c74ec83e7c4c

console.ts:136 [Extension Host] Starting WebSocket: ws://localhost:8888/api/kernels/e4c82ade-2da6-43ae-81c8-c74ec83e7c4c
console.ts:136 [Extension Host] Python Extension: [I 21:00:15.965 NotebookApp] Adapting to protocol v5.1 for kernel e4c82ade-2da6-43ae-81c8-c74ec83e7c4c

console.ts:136 [Extension Host] Kernel: connected (e4c82ade-2da6-43ae-81c8-c74ec83e7c4c)
console.ts:136 [Extension Host] Python Extension: Execute for import pandas as pd
import numpy
%matplotlib inline
import matplotlib.pyplot as plt silently finished.
console.ts:136 [Extension Host] Python Extension: Execute for %cd "/Users/jayanthkoushik/Projects/few-shot" silently finished.
console.ts:136 [Extension Host] Python Extension: [I 21:01:04.981 NotebookApp] 302 GET / (::1) 0.43ms

console.ts:136 [Extension Host] Python Extension: [I 21:01:04.988 NotebookApp] 302 GET /tree? (::1) 0.83ms

[mcdevitts]

I'm also getting this behavior.

[rchiodo]

This is happening because the kernel specs returned by Jupyter can't be matched against the virtual environment. We need to figure out some other way to match them up.

When I tried this the kernel spec returned this for the virtualenv:

{ python3: { name: "python3" display_name: "Python 3" language: "python" argv ["python", "-m", "ipykernel_launcher", "-f", "{connection_file}"] } }

We can possibly better match on the major version here, but we don't have the path to the python being used.

[rchiodo]

You can do this to see what version of python is being used:

import sys
print(sys.version)
print(sys.executable)

[DonJayamanne]

This is happening because the kernel specs returned by Jupyter can't be matched against the virtual environment. We need to figure out some other way to match them up.

Suggestion: compare the paths. The first argument is the path to the executable. Look for kernels that have the same path as the currently selected one.

[rchiodo]

No in this case the first argument is not the full path. It's just "python". Otherwise it would have worked.

[DonJayamanne]

"python". Otherwise it would have worked.

Agreed, that's cuz this kennel is using the default Python environment. And you can get that using python -c print(sys.executable)

[rchiodo]

I'm fixing this by generating a new kernel spec for any environment that we don't have one for.