tox is fooled by __PYVENV_LAUNCHER__

[zsimic]

Environment

• OS: OSX 10.15.+
• pip version: any
• Python version: any
• tox version: any

Description

OSX's system python seems to define the infamous __PYVENV_LAUNCHER__ env var... and that fools tox (like many other tools...). It doesn't matter which version of virtualenv or pip you may have. If tox runs from its own dedicated venv created by an OSX system python (or one that uses a similar approach), your tox will not be usable and will fail with No module named pip.

Expected behavior

tox should work even if the python that was used to create the venv where it lives was OSX's system python.

How to Reproduce

Here's how to reproduce this. In a temp directory, create 2 files:

• a sample tox.ini with these contents:

[tox]
envlist = py{36,37,38}

[testenv]
skip_install = True
commands = python --version

• a small shell script called repro.sh to conveniently reproduce this:

#!/bin/bash -x

rm -rf .tox mytox

/usr/bin/python3 -mvenv mytox
./mytox/bin/python -mpip install tox==3.20.1

./mytox/bin/tox

Now ensure that you have the python versions mentioned in tox.ini, in my case I use pyenv and this is what my which -a python shows:

.../pyenv/versions/3.9.0/bin/python
.../pyenv/versions/3.8.6/bin/python
.../pyenv/versions/3.7.9/bin/python
.../pyenv/versions/3.6.12/bin/python

Run the script bash -x ./repros.sh on an OSX 10.15.6 machine (any version should do), you will see that tox here complains about No module named pip...

The tox target that matches your version of /usr/bin/python3 will work, but the other 2 won't.

Now, if you edit mytox/bin/tox and add these 3 lines:

import os
if "__PYVENV_LAUNCHER__" in os.environ:
    del os.environ["__PYVENV_LAUNCHER__"]

Then you will observe that ./mytox/bin/tox now works fine (for any pyNN version).

[gaborbernat]

This is a CPython venv bug. Fixed by python/cpython#9516. It's also fixed within virtualenv here pypa/virtualenv#1704. So my recommendation will be to use virtualenv over venv when creating the virtual environment to install tox into. I'm not totally clear if tox should patch itself to facilitate a venv bug... 🤔

[zsimic]

I agree this bug is silly (also: no idea what the heck is this __PYVENV_LAUNCHER__ thing is even supposed to do... sounds like someone tried hard to add complexity in there for no good reason...)

However, the bug means that pip install tox does NOT work -> it will work only if you run it in a venv NOT created by the standard venv module...

So I guess, because of this bug, you would explain in the tox install instructions that pip install tox comes with a caveat.

I would simply remove that silly env var and avoid all of the confusion... with a note to get rid of it once we're through this silliness

[zsimic]

Note also that the fix you mention in pypa/virtualenv#1704 does not help. You can see that tox does not work even when it uses the latest virtualenv. The bug occurs before that.

The bug simply means that whoever tries to pip install tox in a venv created by stdlib module venv on OSX, will invariably run into this issue, scratch their head... google around, and possibly submit yet another bug report (there are plenty variants out there as you already know).

I used to work around this issue by installing tox with an old python2 venv just to avoid this issue... Now I want to get rid of py2 entirely, so I took the time to report this bug (I was already working around it for months...).

[zsimic]

The bug seems to occur here exactly: https://github.com/tox-dev/tox/blob/master/src/tox/venv.py#L807

list_dependencies_command is python -m pip freeze by default, and that command fails (with No module named pip) when it is ran while __PYVENV_LAUNCHER__ is defined.

[gaborbernat]

I think this should not be the case anymore with tox 4, and there's no fix on our side other than that.