WSL : Python executable coming from outside... and failing. #3488

asmodehn · 2019-01-28T18:11:39Z

Issue description

Attempting to use pipenv to create a virtualenv on Windows(10) Subsystem Linux (Ubuntu distro), pipenv fails during the virtualenv creation. The issues seems to be that pipenv decides to use a python executable from windows, instead of the one from the WSL distro.

The flow I used to install pipenv on WSL was :

install python3.7 via apt,
install python3-pip via apt,
install latest pip in user env via pip3 install --user --upgrade pip,
install pipenv in user env via pip install --user pipenv. Note user shell must be restarted to get the correct pip.
Note I also have Python 3.7.2 installed on Windows.

Expected result

Pipenv decides to use python 3.7.1 coming from WSL environment, and environment is created with success :

alexv@AlexV-Win10-Oryx:/mnt/c/Users/AlexV/Documents/GitHub/functional-python/01$ pipenv install
Creating a virtualenv for this project…
Pipfile: /mnt/c/Users/AlexV/Documents/GitHub/functional-python/01/Pipfile
Using /usr/bin/python3.7 (3.7.1) to create virtualenv…
⠋ Creating virtual environment...Using base prefix '/usr'
New python executable in /mnt/c/Users/AlexV/Documents/GitHub/functional-python/01/.venv/bin/python3.7
Also creating executable in /mnt/c/Users/AlexV/Documents/GitHub/functional-python/01/.venv/bin/python
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python3.7

✔ Successfully created virtual environment!

Actual result

alexv@AlexV-Win10-Oryx:/mnt/c/Users/AlexV/Documents/GitHub/functional-python/01$ pipenv install -v
Creating a virtualenv for this project…
Pipfile: /mnt/c/Users/AlexV/Documents/GitHub/functional-python/01/Pipfile
Using /mnt/c/Program Files/Python37/python.exe (3.7.2) to create virtualenv…
⠹ Creating virtual environment...Running virtualenv with interpreter /mnt/c/Program Files/Python37/python.exe

✘ Failed creating virtual environment
Traceback (most recent call last):
  File "/home/alexv/.local/bin/pipenv", line 10, in <module>
    sys.exit(cli())
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/cli/command.py", line 254, in install
    editable_packages=state.installstate.editables,
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/core.py", line 1741, in do_install
    pypi_mirror=pypi_mirror,
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/core.py", line 574, in ensure_project
    pypi_mirror=pypi_mirror,
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/core.py", line 506, in ensure_virtualenv
    python=python, site_packages=site_packages, pypi_mirror=pypi_mirror
  File "/home/alexv/.local/lib/python3.6/site-packages/pipenv/core.py", line 935, in do_create_virtualenv
    extra=[crayons.blue("{0}".format(c.err)),]
pipenv.exceptions.VirtualenvCreationException: Failed to create virtual environment.

Steps to replicate

Pipfile :

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pytest = "*"

[requires]
python_version = "3.7"

Workaround

Running instead :
pipenv install --python=/usr/bin/python3.7
fixes the problem by specifying which python needs to be used.

$ pipenv --support

Pipenv version: '2018.11.26'

Pipenv location: '/home/alexv/.local/lib/python3.6/site-packages/pipenv'

Python location: '/usr/bin/python3'

Python installations found:

3.7.2: /mnt/c/Program Files/Python37/python.exe
3.7.2: /mnt/c/Program Files/Python37/pythonw.exe
3.7.1: /usr/bin/python3.7
3.7.1: /usr/bin/python3.7m
3.6.7: /usr/bin/python3
3.6.7: /usr/bin/python3.6m

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.7',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.4.0-17763-Microsoft',
 'platform_system': 'Linux',
 'platform_version': '#253-Microsoft Mon Dec 31 17:49:00 PST 2018',
 'python_full_version': '3.6.7',
 'python_version': '3.6',
 'sys_platform': 'linux'}

System environment variables:

LS_COLORS
HOSTTYPE
LESSCLOSE
LANG
DIRENV_DIR
OLDPWD
DIRENV_DIFF
VIRTUAL_ENV
USER
DIRENV_WATCHES
PWD
HOME
NAME
XDG_DATA_DIRS
PIPENV_ACTIVE
SHELL
TERM
SHLVL
LOGNAME
PATH
WSLENV
LESSOPEN
BASH_FUNC_show_virtual_env%%
_
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_SHIMS_BASE_MODULE
PIP_PYTHON_PATH
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

PIPENV_ACTIVE: 1

Debug–specific environment variables:

PATH: /bin:/home/alexv/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Program Files/Python37/Scripts:/mnt/c/Program Files/Python37:/mnt/c/Windows/System32:/mnt/c/Windows:/mnt/c/Windows/System32/wbem:/m nt/c/Windows/System32/WindowsPowerShell/v1.0:/mnt/c/Windows/System32/OpenSSH:/mnt/c/ProgramData/chocolatey/bin:/mnt/c/Program Files/NVIDIA Corporation/NVIDIA NvDLISR:/mnt/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/mnt/c/Program Files/Git/cmd:/mnt/c/Users/A lexV/AppData/Roaming/Python/Python37/Scripts:/mnt/c/tools/Cmder:/mnt/c/tools/direnv:/mnt/c/tools/Syncthing:/mnt/c/Users/AlexV/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/AlexV/AppData/Local/GitHubDesktop/bin:/snap/bin
SHELL: /bin/bash
LANG: C.UTF-8
PWD: /mnt/c/Users/AlexV/Documents/GitHub/functional-python/01
VIRTUAL_ENV: ``

Contents of Pipfile ('/mnt/c/Users/AlexV/Documents/GitHub/functional-python/01/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pytest = "*"

[requires]
python_version = "3.7"

Contents of Pipfile.lock ('/mnt/c/Users/AlexV/Documents/GitHub/functional-python/01/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "828b8ad012f4c8773e6e61e3ac2be0ffcd7540fd7ed175a8355676c8e31c4d3d"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "atomicwrites": {
            "hashes": [
                "sha256:0312ad34fcad8fac3704d441f7b317e50af620823353ec657a53e981f92920c0",
                "sha256:ec9ae8adaae229e4f8446952d204a3e4b5fdd2d099f9be3aaf556120135fb3ee"
            ],
            "version": "==1.2.1"
        },
        "attrs": {
            "hashes": [
                "sha256:10cbf6e27dbce8c30807caf056c8eb50917e0eaafe86347671b57254006c3e69",
                "sha256:ca4be454458f9dec299268d472aaa5a11f67a4ff70093396e1ceae9c76cf4bbb"
            ],
            "version": "==18.2.0"
        },
        "more-itertools": {
            "hashes": [
                "sha256:38a936c0a6d98a38bcc2d03fdaaedaba9f412879461dd2ceff8d37564d6522e4",
                "sha256:c0a5785b1109a6bd7fac76d6837fd1feca158e54e521ccd2ae8bfe393cc9d4fc",
                "sha256:fe7a7cae1ccb57d33952113ff4fa1bc5f879963600ed74918f1236e212ee50b9"
            ],
            "version": "==5.0.0"
        },
        "pluggy": {
            "hashes": [
                "sha256:8ddc32f03971bfdf900a81961a48ccf2fb677cf7715108f85295c67405798616",
                "sha256:980710797ff6a041e9a73a5787804f848996ecaa6f8a1b1e08224a5894f2074a"
            ],
            "version": "==0.8.1"
        },
        "py": {
            "hashes": [
                "sha256:bf92637198836372b520efcba9e020c330123be8ce527e535d185ed4b6f45694",
                "sha256:e76826342cefe3c3d5f7e8ee4316b80d1dd8a300781612ddbc765c17ba25a6c6"
            ],
            "version": "==1.7.0"
        },
        "pytest": {
            "hashes": [
                "sha256:41568ea7ecb4a68d7f63837cf65b92ce8d0105e43196ff2b26622995bb3dc4b2",
                "sha256:c3c573a29d7c9547fb90217ece8a8843aa0c1328a797e200290dc3d0b4b823be"
            ],
            "index": "pypi",
            "version": "==4.1.1"
        },
        "six": {
            "hashes": [
                "sha256:3350809f0555b11f552448330d0b52d5f24c91a322ea4a15ef22629740f3761c",
                "sha256:d16a0141ec1a18405cd4ce8b4613101da75da0e9a7aec5bdd4fa804d0e0eba73"
            ],
            "version": "==1.12.0"
        }
    },
    "develop": {}
}

The text was updated successfully, but these errors were encountered:

patrickayoup · 2019-05-21T04:55:14Z

I had the same issue on WSL, I noticed something interesting.
My Pipfile requires 3.7 but I didn't have 3.7 installed on the subsystem. After installing it, pipenv install worked without any issue and didn't try to go to the windows executable anymore.

tony · 2019-05-25T17:02:33Z

pipenv install --python=/usr/bin/python3.7 does fix for me

pipenv is also pulling up python.exe for me

Config / settings

Ubuntu WSL, Windows 1903, 
❯ sudo lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.2 LTS
Release:        18.04
Codename:       bionic

❯ pip --version
pip 19.1.1 from /home/x/.local/lib/python3.6/site-packages/pip (python 3.6)

❯ pipenv --version
pipenv, version 2018.11.26

❯ python --version
Python 2.7.15rc1

❯ python3 --version
Python 3.6.7

❯ python3.7 --version
Python 3.7.1

I also bumped into this issue from inside a WSL environment.

I have python 3.7 installed, but pip is installing to my python3.6 site packages by default.

I only specify python 3 in my Pipfile:

[requires]
python_version = "3"

Error

Error output:
['Traceback (most recent call last):\n', '  File "/home/x/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 160, in _create_subprocess\n    combine_stderr=combine_stderr)\n', '  File "/home/x/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 134, in _spawn_subprocess\n    return subprocess.Popen(cmd, **options)\n', '  File "/usr/lib/python3.6/subprocess.py", line 709, in __init__\n    restore_signals, start_new_session)\n', '  File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child\n    raise child_exception_type(errno_num, err_msg, err_filename)\n', "OSError: [Errno 8] Exec format error: '/mnt/c/Users/tony/AppData/Local/Microsoft/WindowsApps/python.exe'\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', '  File "/home/x/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/contextmanagers.py", line 150, in spinner\n    yield _spinner\n', '  File "/home/x/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 314, in run\n    write_to_stdout=True\n', '  File "/home/x/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 162, in _create_subprocess\n    sys.stderr.write("Error %s while executing command %s", exc, " ".join(cmd._parts))\n', 'TypeError: write() takes 2 positional arguments but 4 were given\n']
Traceback (most recent call last):
  File "/home/x/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 160, in _create_subprocess
    combine_stderr=combine_stderr)
  File "/home/x/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 134, in _spawn_subprocess
    return subprocess.Popen(cmd, **options)
  File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/mnt/c/Users/tony/AppData/Local/Microsoft/WindowsApps/python.exe'
During handling of the above exception, another exception occurred:

thernstig · 2019-06-21T09:35:57Z

I ran into this as well today. Very strange that it looks for the Python in the Windows path.

frostming · 2019-06-21T13:10:35Z

The python version on Windows is higher than that on WSL, making it come before the latter one..

Specify python path explicitly can fix the problem:

$ pipenv --python /usr/bin/python3

thernstig · 2019-06-22T10:03:32Z

@frostming I am unsure why you closed this, can you elaborate? This is a real concern that you most likely will get a large influx of issues on. This is a serious bug that will hit anyone developing in WSL.

In the Windows 10, May 2019 update (19.03) python.exe and python3.exe are found by default, but they do not point to a real Python executable. I.e. a user have these without having installed Python in Windows. Read this announcement https://devblogs.microsoft.com/python/python-in-the-windows-10-may-2019-update/ and more specifically this quote:

Finally, with the May 2019 Windows Update, we are completing the picture. While Python continues to remain completely independent from the operating system, every install of Windows will include python and python3 commands that take you directly to the Python store page.

I get this failure even though I have only installed python into the WSL distribution, and my project resides in the /home/username/ dir.

11:56:55 computer1 ~/code/scrapy (master)$ which python
/usr/bin/python
11:57:01 computer1 ~/code/scrapy (master)$ which python3
/usr/bin/python3
11:57:04 computer1 ~/code/scrapy (master)$ printenv PATH
/home/username/code/scrapy/.venv/bin:/home/username/.pyenv/shims:/home/username/.pyenv/bin:/home/username/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Program Files/WindowsApps/CanonicalGroupLimited.UbuntuonWindows_1804.2019.521.0_x64__79rhkp1fndgsc:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0/:/mnt/c/WINDOWS/System32/OpenSSH/:/mnt/c/Users/username/AppData/Local/Microsoft/WindowsApps:/snap/bin

This completely breaks working in WSL and automatic installing like with pyenv, or using direnv with layout_pipenv. And the user on the 2019 May update has not even installed in Windows.

frostming · 2019-06-22T10:15:08Z

@thernstig I didn't say this is not an issue. Let's track it in #3807

There should be a fix in pythonfinder projects and the vendored library.

shamoons · 2019-10-15T23:56:32Z

How do you maintain multiple Python versions? Specifically 3.6 and 3.7?

yuri-stepanov · 2019-11-17T17:23:10Z

@shamoons I personally use pyenv for that. (https://github.com/pyenv/pyenv)

mjherich · 2019-12-06T03:01:26Z

I got annoyed from having to specify my python version every time so I just removed %USERPROFILE%\AppData\Local\Microsoft\WindowsApps from PATH in Windows. Now pipenv never gets confused with the python.exe that Micrsoft put in there.

danmichaelo · 2019-12-17T12:43:46Z

I got annoyed from having to specify my python version every time so I just removed %USERPROFILE%\AppData\Local\Microsoft\WindowsApps from PATH in Windows. Now pipenv never gets confused with the python.exe that Micrsoft put in there.

Also works to just disable the app execution alias (Settings > Apps > Apps & features > Manage app execution aliases): https://superuser.com/a/1461471/56154

markfaine mentioned this issue Jan 29, 2019

WSL: pipenv shell does not launch a subshell #3489

Closed

rjnfrazao mentioned this issue May 28, 2019

How to deactivate current environment entered by pipenv shell? #84

Closed

frostming added Type: Bug 🐛 This issue is a bug. Type: Vendored Dependencies This issue affects vendored dependencies within pipenv. labels Jun 21, 2019

frostming closed this as completed Jun 21, 2019

frostming mentioned this issue Jun 22, 2019

Can't find the python path with python3 #3807

Closed

g1augusto mentioned this issue Sep 22, 2020

April (then May) 2020 Release Tracking Issue #3369

Closed

30 tasks

fabmeyer mentioned this issue Feb 28, 2022

PermissionError: [Errno 13] Permission denied: '/mnt/c/windows/system32/config/systemprofile/AppData/Local/Microsoft/WindowsApps' microsoft/WSL#8100

Closed

2 tasks

r7kamura mentioned this issue Jun 19, 2022

2022-06-20 r7kamura/diary#45

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WSL : Python executable coming from outside... and failing. #3488

WSL : Python executable coming from outside... and failing. #3488

asmodehn commented Jan 28, 2019

patrickayoup commented May 21, 2019

tony commented May 25, 2019 •

edited

Loading

thernstig commented Jun 21, 2019

frostming commented Jun 21, 2019 •

edited

Loading

thernstig commented Jun 22, 2019 •

edited

Loading

frostming commented Jun 22, 2019

shamoons commented Oct 15, 2019

yuri-stepanov commented Nov 17, 2019

mjherich commented Dec 6, 2019

danmichaelo commented Dec 17, 2019

WSL : Python executable coming from outside... and failing. #3488

WSL : Python executable coming from outside... and failing. #3488

Comments

asmodehn commented Jan 28, 2019

Issue description

Expected result

Actual result

Steps to replicate

Workaround

patrickayoup commented May 21, 2019

tony commented May 25, 2019 • edited Loading

thernstig commented Jun 21, 2019

frostming commented Jun 21, 2019 • edited Loading

thernstig commented Jun 22, 2019 • edited Loading

frostming commented Jun 22, 2019

shamoons commented Oct 15, 2019

yuri-stepanov commented Nov 17, 2019

mjherich commented Dec 6, 2019

danmichaelo commented Dec 17, 2019

tony commented May 25, 2019 •

edited

Loading

frostming commented Jun 21, 2019 •

edited

Loading

thernstig commented Jun 22, 2019 •

edited

Loading