python2.7 sysconfig broken under virtualenv #118

bukzor · 2011-04-13T04:45:05Z

This is preventing numpy from installing correctly:

$ virtualenv --version
1.6

>>> import sysconfig
>>> sysconfig.get_config_vars()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/sysconfig.py", line 444, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File "/usr/lib/python2.7/sysconfig.py", line 298, in _init_posix
    raise IOError(msg)
IOError: invalid Python installation: unable to open /home/bgolemon/software/sqlalchemy/local/lib/python2.7/config/Makefile (No such file or directory)

>>> sysconfig._get_makefile_filename()
'/home/bgolemon/software/sqlalchemy/local/lib/python2.7/config/Makefile'

Quick fix:
mkdir local
ln -s ../lib local/lib

The text was updated successfully, but these errors were encountered:

bukzor · 2011-04-13T04:51:42Z

The culprit is this line at /usr/lib/python2.7/sysconfig.py:285

return os.path.join(get_path('stdlib').replace("/usr/local","/usr",1), "config" + (sys.pydebug and "_d" or ""), "Makefile")

There is no '/usr' in the stdlib path under virtualenv, so the string replace above doesn't do anything. I don't know whose bug this is really, or how best to fix it, so I'll leave it up to you all from here.

pshields · 2011-05-22T22:12:09Z

I encountered this issue as well. The previous poster's quick fix worked for me as a temporary solution.

scottza · 2011-06-01T08:38:15Z

It looks like the function path_locations() in virtualenv.py needs to be special cased for Debian based systems with Python 2.7.

On my Ubuntu 11.04 machine

import sysconfig
print(sysconfig.get_paths())
{'platinclude': '/usr/local/include/python2.7', 'platstdlib': '/usr/local/lib/python2.7', 'platlib': '/usr/local/lib/python2.7/dist-packages', 'purelib': '/usr/local/lib/python2.7/dist-packages', 'stdlib': '/usr/local/lib/python2.7', 'scripts': '/usr/local/bin', 'include': '/usr/local/include/python2.7', 'data': '/usr/local'}

virtualenv puts stuff in $VIRTUAL_ENV/lib, $VIRTUAL_ENV/include etc.

My version of bukzor's workaround is:

Activate the virtualenv
mkdir $VIRTUAL_ENV/local
ln -s $VIRTUAL_ENV/lib $VIRTUAL_ENV/local/lib
Install Numpy

wgx731 · 2011-06-08T14:46:17Z

I want to install numpy with virtualenv on Ubuntu 11.04 as well. Encounter error. This is my log:
Downloading/unpacking numpy Running setup.py egg_info for package numpy Running from numpy source directory./bin/sh: svnversion: not found non-existing path in 'numpy/distutils': 'site.cfg' F2PY Version 2 blas_opt_info: blas_mkl_info: libraries mkl,vml,guide not found in /home/wgx731/.virtualenvs/python2.7/lib libraries mkl,vml,guide not found in /usr/local/lib libraries mkl,vml,guide not found in /usr/lib NOT AVAILABLE


atlas_blas_threads_info:
Setting PTATLAS=ATLAS
  libraries ptf77blas,ptcblas,atlas not found in /home/wgx731/.virtualenvs/python2.7/lib
  libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib
  NOT AVAILABLE

atlas_blas_info:
  libraries f77blas,cblas,atlas not found in /home/wgx731/.virtualenvs/python2.7/lib
  libraries f77blas,cblas,atlas not found in /usr/local/lib
  libraries f77blas,cblas,atlas not found in /usr/lib/sse2
  libraries f77blas,cblas,atlas not found in /usr/lib
  NOT AVAILABLE

/home/wgx731/.virtualenvs/python2.7/build/numpy/numpy/distutils/system_info.py:1417: UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
blas_info:
  libraries blas not found in /home/wgx731/.virtualenvs/python2.7/lib
  libraries blas not found in /usr/local/lib
  libraries blas not found in /usr/lib
  NOT AVAILABLE

/home/wgx731/.virtualenvs/python2.7/build/numpy/numpy/distutils/system_info.py:1426: UserWarning:
    Blas (http://www.netlib.org/blas/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [blas]) or by setting
    the BLAS environment variable.
  warnings.warn(BlasNotFoundError.__doc__)
blas_src_info:
  NOT AVAILABLE

/home/wgx731/.virtualenvs/python2.7/build/numpy/numpy/distutils/system_info.py:1429: UserWarning:
    Blas (http://www.netlib.org/blas/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [blas_src]) or by setting
    the BLAS_SRC environment variable.
  warnings.warn(BlasSrcNotFoundError.__doc__)
  NOT AVAILABLE

/bin/sh: svnversion: not found
lapack_opt_info:
lapack_mkl_info:
mkl_info:
  libraries mkl,vml,guide not found in /home/wgx731/.virtualenvs/python2.7/lib
  libraries mkl,vml,guide not found in /usr/local/lib
  libraries mkl,vml,guide not found in /usr/lib
  NOT AVAILABLE

  NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
  libraries ptf77blas,ptcblas,atlas not found in /home/wgx731/.virtualenvs/python2.7/lib
  libraries lapack_atlas not found in /home/wgx731/.virtualenvs/python2.7/lib
  libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/local/lib
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
  libraries lapack_atlas not found in /usr/lib/sse2
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib
  libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_threads_info
  NOT AVAILABLE

atlas_info:
  libraries f77blas,cblas,atlas not found in /home/wgx731/.virtualenvs/python2.7/lib
  libraries lapack_atlas not found in /home/wgx731/.virtualenvs/python2.7/lib
  libraries f77blas,cblas,atlas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/local/lib
  libraries f77blas,cblas,atlas not found in /usr/lib/sse2
  libraries lapack_atlas not found in /usr/lib/sse2
  libraries f77blas,cblas,atlas not found in /usr/lib
  libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_info
  NOT AVAILABLE

/home/wgx731/.virtualenvs/python2.7/build/numpy/numpy/distutils/system_info.py:1333: UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
lapack_info:
  libraries lapack not found in /home/wgx731/.virtualenvs/python2.7/lib
  libraries lapack not found in /usr/local/lib
  libraries lapack not found in /usr/lib
  NOT AVAILABLE

/home/wgx731/.virtualenvs/python2.7/build/numpy/numpy/distutils/system_info.py:1344: UserWarning:
    Lapack (http://www.netlib.org/lapack/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [lapack]) or by setting
    the LAPACK environment variable.
  warnings.warn(LapackNotFoundError.__doc__)
lapack_src_info:
  NOT AVAILABLE

/home/wgx731/.virtualenvs/python2.7/build/numpy/numpy/distutils/system_info.py:1347: UserWarning:
    Lapack (http://www.netlib.org/lapack/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [lapack_src]) or by setting
    the LAPACK_SRC environment variable.
  warnings.warn(LapackSrcNotFoundError.__doc__)
  NOT AVAILABLE

build_src
building py_modules sources
building library "npymath" sources
error: None
Complete output from command python setup.py egg_info:
Running from numpy source directory./bin/sh: svnversion: not found

non-existing path in 'numpy/distutils': 'site.cfg'
F2PY Version 2
blas_opt_info:
blas_mkl_info:
libraries mkl,vml,guide not found in /home/wgx731/.virtualenvs/python2.7/lib
libraries mkl,vml,guide not found in /usr/local/lib
libraries mkl,vml,guide not found in /usr/lib
NOT AVAILABLE
atlas_blas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in /home/wgx731/.virtualenvs/python2.7/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
NOT AVAILABLE
atlas_blas_info:
libraries f77blas,cblas,atlas not found in /home/wgx731/.virtualenvs/python2.7/lib
libraries f77blas,cblas,atlas not found in /usr/local/lib
libraries f77blas,cblas,atlas not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib
NOT AVAILABLE
/home/wgx731/.virtualenvs/python2.7/build/numpy/numpy/distutils/system_info.py:1417: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.

Directories to search for the libraries can be specified in the

numpy/distutils/site.cfg file (section [atlas]) or by setting

the ATLAS environment variable.

warnings.warn(AtlasNotFoundError.doc)
blas_info:
libraries blas not found in /home/wgx731/.virtualenvs/python2.7/lib
libraries blas not found in /usr/local/lib
libraries blas not found in /usr/lib
NOT AVAILABLE
/home/wgx731/.virtualenvs/python2.7/build/numpy/numpy/distutils/system_info.py:1426: UserWarning:
Blas (http://www.netlib.org/blas/) libraries not found.

Directories to search for the libraries can be specified in the

numpy/distutils/site.cfg file (section [blas]) or by setting

the BLAS environment variable.

warnings.warn(BlasNotFoundError.doc)
blas_src_info:
NOT AVAILABLE
/home/wgx731/.virtualenvs/python2.7/build/numpy/numpy/distutils/system_info.py:1429: UserWarning:
Blas (http://www.netlib.org/blas/) sources not found.

Directories to search for the sources can be specified in the

numpy/distutils/site.cfg file (section [blas_src]) or by setting

the BLAS_SRC environment variable.

warnings.warn(BlasSrcNotFoundError.doc)
NOT AVAILABLE
/bin/sh: svnversion: not found
lapack_opt_info:
lapack_mkl_info:
mkl_info:
libraries mkl,vml,guide not found in /home/wgx731/.virtualenvs/python2.7/lib
libraries mkl,vml,guide not found in /usr/local/lib
libraries mkl,vml,guide not found in /usr/lib
NOT AVAILABLE
NOT AVAILABLE
atlas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in /home/wgx731/.virtualenvs/python2.7/lib
libraries lapack_atlas not found in /home/wgx731/.virtualenvs/python2.7/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/local/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
libraries lapack_atlas not found in /usr/lib/sse2
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_threads_info
NOT AVAILABLE
atlas_info:
libraries f77blas,cblas,atlas not found in /home/wgx731/.virtualenvs/python2.7/lib
libraries lapack_atlas not found in /home/wgx731/.virtualenvs/python2.7/lib
libraries f77blas,cblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/local/lib
libraries f77blas,cblas,atlas not found in /usr/lib/sse2
libraries lapack_atlas not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_info
NOT AVAILABLE
/home/wgx731/.virtualenvs/python2.7/build/numpy/numpy/distutils/system_info.py:1333: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.

Directories to search for the libraries can be specified in the

numpy/distutils/site.cfg file (section [atlas]) or by setting

the ATLAS environment variable.

warnings.warn(AtlasNotFoundError.doc)
lapack_info:
libraries lapack not found in /home/wgx731/.virtualenvs/python2.7/lib
libraries lapack not found in /usr/local/lib
libraries lapack not found in /usr/lib
NOT AVAILABLE
/home/wgx731/.virtualenvs/python2.7/build/numpy/numpy/distutils/system_info.py:1344: UserWarning:
Lapack (http://www.netlib.org/lapack/) libraries not found.

Directories to search for the libraries can be specified in the

numpy/distutils/site.cfg file (section [lapack]) or by setting

the LAPACK environment variable.

warnings.warn(LapackNotFoundError.doc)
lapack_src_info:
NOT AVAILABLE
/home/wgx731/.virtualenvs/python2.7/build/numpy/numpy/distutils/system_info.py:1347: UserWarning:
Lapack (http://www.netlib.org/lapack/) sources not found.

Directories to search for the sources can be specified in the

numpy/distutils/site.cfg file (section [lapack_src]) or by setting

the LAPACK_SRC environment variable.

warnings.warn(LapackSrcNotFoundError.doc)
NOT AVAILABLE
running egg_info
running build_src
build_src
building py_modules sources
building library "npymath" sources
error: None

Command python setup.py egg_info failed with error code 1
I have tried scottza's solution, but it doesn't work for me.
Any way to fix this? Thx

pnasrat · 2011-06-08T21:26:01Z

Thanks, for the update. We're aware of this issue and I hope to look at in more depth this weekend.

flaviovdf · 2011-06-25T21:37:00Z

Managed to get it working with the following workaround:

mkdir $VIRTUAL_ENV/local
ln -s $VIRTUAL_ENV/lib $VIRTUAL_ENV/local/lib
ln -s $VIRTUAL_ENV/include $VIRTUAL_ENV/local/include

lofidevops · 2011-09-15T06:02:40Z

Reported fix to https://bugs.launchpad.net/ubuntu/+source/python-virtualenv/+bug/820792

robertmuil · 2012-03-15T14:00:41Z

This fix is broken in certain setups.

If the python virtualenv path includes ...usr/local... then the virtualenv setup script will erroneously 'correct' it to ...local...

So, moving the virtualenv will avoid the problem: worth knowing for anyone who likes to install things under $HOME/usr.

mikofski · 2012-04-02T08:33:43Z

still can't install numpy under virtualenv, but I also couldn't install it with pip just in system env.
virtualenv 1.6.4
ubuntu 11.10
pip 1.0
distribute 0.6.16

dmckean · 2013-08-17T00:18:49Z

Something in sysconfig in python 2.7 seems to be at fault.

Python 2.7.3 (default, Apr 10 2013, 06:20:15)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_vars()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/sysconfig.py", line 476, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File "/usr/lib/python2.7/sysconfig.py", line 344, in _init_posix
    raise IOError(msg)
IOError: invalid Python installation: unable to open /usr/django/app/my_env/lib/python2.7/config/Makefile (No such file or directory)
>>> sysconfig.get_config_vars()
{'py_version_nodot': '27', 'exec_prefix': '/usr/local/django/app/my_env', 'platbase': '/usr/local/django/app/my_env', 'py_version_short': '2.7', 'prefix': '/usr/local/django/app/my_env', 'base': '/usr/local/django/app/my_env', 'projectbase': '/usr/local/django/app/my_env/bin', 'py_version': '2.7.3'}
>>>

nicksloan · 2013-10-18T00:15:27Z

This issue is caused by a patch applied by Ubuntu that specifically replaced /usr/local with /usr in several functions. Line 63404 in https://launchpadlibrarian.net/151545955/python2.7_2.7.3-5ubuntu4.3.diff.gz.

This stinks of taking a shortcut to support Ubuntu's deviations from the typical posix_local schema. Annoying.

For future reference: moving your virtualenv out of /usr/local... will fix it.

brianz · 2013-11-21T21:53:34Z

Hit this on Ubunu 12.04 and pip 1.4.1, virtualenv 1.10.1....moving the virtualenv out of /usr/local did the trick.

Neetuj · 2014-04-25T19:56:45Z

running into the same issue
root@njain2:/usr/local/horizon# pip --version
pip 1.5.4 from /usr/local/lib/python2.7/dist-packages (python 2.7)
root@njain2:/usr/local/horizon# python --version
Python 2.7.3
root@njain2:/usr/local/horizon# virtualenv --version
1.7.1.2

File "/usr/lib/python2.7/sysconfig.py", line 344, in _init_posix

raise IOError(msg)
IOError: invalid Python installation: unable to open /usr/horizon/.venv/lib/python2.7/config/Makefile (No such file or directory)

Cleaning up...
Command python setup.py egg_info failed with error code 1 in /usr/local/horizon/.venv/build/cryptography
Storing debug log for failure in /root/.pip/pip.log

Neetuj · 2014-04-25T20:13:32Z

the file exists in /usr/local ..but i am not sure how to make it read from there .. and where exaclty it should be

root@njain2:/usr/local/horizon# ls /usr/horizon/.venv/lib/python2.7/config/Makefile
ls: cannot access /usr/horizon/.venv/lib/python2.7/config/Makefile: No such file or directory
root@njain2:/usr/local/horizon# ls /usr/local/horizon/.venv/lib/python2.7/config/Makefile
/usr/local/horizon/.venv/lib/python2.7/config/Makefile

pkpp1233 · 2014-07-11T07:53:21Z

i'm running into this same issue trying to install numpy in a docker container.

bukzor · 2014-11-14T17:50:07Z

@Neetuj: The culprit for you is still this line at /usr/lib/python2.7/sysconfig.py:285, shown above, although for a different reason.

bukzor · 2014-11-14T19:48:40Z

This was apparently an ill-conceived patch released by ubuntu for precise:
http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/precise/python2.7/precise-security/view/head:/debian/patches/distutils-install-layout.diff#L282

The replace statement is gone in trusty, but the scheme is still set to 'posix-local', so virtualenv's workaround persists, causing bugs in the coveragepy test suite under ubuntu:

https://bitbucket.org/ned/coveragepy/issue/337/test-failures-on-tip#comment-13653960

naarkhoo · 2014-11-25T13:22:44Z

I have the same problem - for installing Pilow on virtualenv

creating pip-egg-info/Pillow.egg-info

writing pip-egg-info/Pillow.egg-info/PKG-INFO

writing top-level names to pip-egg-info/Pillow.egg-info/top_level.txt

writing dependency_links to pip-egg-info/Pillow.egg-info/dependency_links.txt

writing manifest file 'pip-egg-info/Pillow.egg-info/SOURCES.txt'

warning: manifest_maker: standard file '-c' not found

error: invalid Python installation: unable to open /usr/lib/python2.7/config/Makefile (No such file or directory)

Cleaning up...
Command python setup.py egg_info failed with error code 1 in /home/rostam/.virtualenvs/myPelican/build/Pillow
Storing debug log for failure in /home/rostam/.pip/pip.log

Is there any workaround ?

bukzor closed this as completed Apr 13, 2011

bukzor reopened this Apr 13, 2011

jezdez closed this as completed in 285679c Jul 16, 2011

dmckean mentioned this issue Aug 17, 2013

Unable to find venv/lib/python2.7/config/Makefile pypa/pip#1074

Closed

This was referenced Dec 10, 2016

Skip "fix_local_scheme" for "virtualenv ."? (symlinks all files) #617

Closed

Virtualenv looks at system site packages (Ubuntu 16.04) #995

Closed

nedbat mentioned this issue Jun 23, 2018

KeyError: sub.py nedbat/coveragepy#340

Closed

cjerdonek mentioned this issue Aug 12, 2018

pip uninstall not working with editable dependencies pypa/pip#5193

Closed

pypa locked and limited conversation to collaborators Jan 14, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

python2.7 sysconfig broken under virtualenv #118

python2.7 sysconfig broken under virtualenv #118

bukzor commented Apr 13, 2011

bukzor commented Apr 13, 2011

pshields commented May 22, 2011

scottza commented Jun 1, 2011

wgx731 commented Jun 8, 2011

pnasrat commented Jun 8, 2011

flaviovdf commented Jun 25, 2011

lofidevops commented Sep 15, 2011

robertmuil commented Mar 15, 2012

mikofski commented Apr 2, 2012

dmckean commented Aug 17, 2013

nicksloan commented Oct 18, 2013

brianz commented Nov 21, 2013

Neetuj commented Apr 25, 2014

Neetuj commented Apr 25, 2014

pkpp1233 commented Jul 11, 2014

bukzor commented Nov 14, 2014

bukzor commented Nov 14, 2014

naarkhoo commented Nov 25, 2014

python2.7 sysconfig broken under virtualenv #118

python2.7 sysconfig broken under virtualenv #118

Comments

bukzor commented Apr 13, 2011

bukzor commented Apr 13, 2011

pshields commented May 22, 2011

scottza commented Jun 1, 2011

wgx731 commented Jun 8, 2011

pnasrat commented Jun 8, 2011

flaviovdf commented Jun 25, 2011

lofidevops commented Sep 15, 2011

robertmuil commented Mar 15, 2012

mikofski commented Apr 2, 2012

dmckean commented Aug 17, 2013

nicksloan commented Oct 18, 2013

brianz commented Nov 21, 2013

Neetuj commented Apr 25, 2014

Neetuj commented Apr 25, 2014

pkpp1233 commented Jul 11, 2014

bukzor commented Nov 14, 2014

bukzor commented Nov 14, 2014

naarkhoo commented Nov 25, 2014