Broken activation in Windows for python3

[svillar]

virtualenv activation through activate_this.py is broken in Windows for python3. Check this very simple piece of code

import os
import subprocess
import sys
from distutils.spawn import find_executable

venv = find_executable("virtualenv")
testdir = os.path.join(os.path.curdir, 'testenv')

subprocess.check_output((venv, testdir, "-p", sys.executable))

bin_path = os.path.join(testdir, "Scripts") if sys.platform in ("win32", "cygwin") else os.path.join(testdir, "bin")

path = os.path.join(bin_path, "activate_this.py")
with open(path) as f:
    exec(f.read(), {"__file__": path})

This generates a AttributeError: 'str' object has no attribute 'decode'. Taking a look at the activate_this.py code:

for lib in "..\Lib\site-packages".split(os.pathsep):
    path = os.path.realpath(os.path.join(bin_dir, lib))
    site.addsitedir(path.decode("utf-8") if "yes" else path)

it's indeed normal that we get the error because we're always calling decode() over a str. Question is where the __DECODE_PATH__ from https://github.com/pypa/virtualenv/blob/master/src/virtualenv/activation/python/activate_this.py#L28 is assigned to "yes"?

[asottile]

it comes from here (it gets replaced):

virtualenv/src/virtualenv/activation/python/__init__.py

Line 23 in 52e4213

"__DECODE_PATH__": ("yes" if WIN_CPYTHON_2 else ""),

it should probably be substituted as a boolean or == "yes" as a fix

[asottile]

actually no, it looks correct as written, that'll only be truthy in cpython2.x windows (assuming the constant is correct)

[asottile]

ah, but the constant is for the host python, not the destination python

[svillar]

Yeah so the thing is why is it "yes" when executing the above script using python3 ?

[asottile]

in this case it's "yes" because you used a python2.x virtualenv executable to create a python3.x virtualenv -- the constant shouldn't be using the host python when templating that file is the bug

you can work around by using a python3.x virtualenv executable to create your virtualenv

[svillar]

well I'm using py -3 to execute it, not sure why is invoking the py2 virtualenv... my windows knowledge is pretty limitted though

[asottile]

right, it's not about the python that executes that script, but the python that was used to create the virtualenv

to be clear, there's a bug in virtualenv here -- but a workaround is to use the same python version when creating the virtualenv

[svillar]

Ah yeah, I've just skimmed through the PATH variable in windows and it turns out that py2.7 is listed before than py3 so find_executable() indeed locates the py2 virtualenv executable

[svillar]

Just for reference I think I can workaround it by doing something like

search_path = os.path.dirname(sys.executable)
find_executable('virtualenv', search_path)

and for the case of windows I guess I should concat Scripts to the search_path

[gaborbernat]

Hello, a fix for this issue has been released via virtualenv 20.0.11; see https://pypi.org/project/virtualenv/20.0.11/ (https://virtualenv.pypa.io/en/latest/changelog.html#v20-0-11-2020-03-18). Please give a try and report back if your issue has not been addressed; if not, please comment here, and we'll reopen the ticket. We want to apologize for the inconvenience this has caused you and say thanks for having patience while we resolve the unexpected bugs with this new major release.