Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Python 2.7 framework-style distributions on macOS 12 #2325

Merged
merged 1 commit into from
Apr 11, 2022

Conversation

nickhutchinson
Copy link
Contributor

@nickhutchinson nickhutchinson commented Mar 29, 2022

virtualenv's trick of adding a symlink to the base Python dylib (/Library/Frameworks/Python.framework/Versions/2.7/Python) in the virtualenv root, and rewriting the dependent dylib list of the <virtualenv>/bin/python binary with the path of this symlink, no longer works in macOS 12.

As of macOS 12, when https://github.com/python/cpython/blob/v2.7.18/Modules/getpath.c computes sys.prefix, it ends up resolving the symlink. The result is that sys.prefix points to the Python framework, e.g. /path/to/Python.framework/Versions/2.7 instead of at the virtualenv root.

To mitigate this, add a copy of the Python dylib into the virtualenv instead of a symlink. To placate code signing (at least on Intel machines), add a symlink to the Resources dir so the Info.plist file is accessible.

Note that Apple's Python 2.7 distribution -- removed in macOS 12.3 but previously located at /System/Library/Frameworks/Python.framework -- behaved differently to the python.org builds. It appears to have incorporated custom patches to getpath.c to support virtualenv. See: https://github.com/apple-oss-distributions/python/blob/python-170.80.2/2.7/fix/getpath.c.ed.

Tested on macOS 12.3 on an Intel and M1 mac.

Fixes: #2284

Thanks for contributing, make sure you address all the checklists (for details on how see

development documentation)!

  • ran the linter to address style issues (tox -e fix_lint)
  • wrote descriptive pull request text
  • ensured there are test(s) validating the fix
  • added news fragment in docs/changelog folder
  • updated/extended the documentation

@@ -0,0 +1 @@
Fix for creating virtualenvs from a Python 2.7 framework on macOS 12.3 - by :user:`nickhutchinson`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I consider this a feature rather than a bugfix, so let's move it to a feature file 👍 something like support for macos 12.3 👍

@nickhutchinson
Copy link
Contributor Author

I want to try and test in a macOS 12.2 VM -- rereading the original thread more carefully, it might be that the behaviour change was actually in macOS 12.0, and Apple's Python 2 (removed in macOS 12.3) had some custom patches to work around the issue.

@nickhutchinson nickhutchinson changed the title Fix for Python 2 framework support on macOS 12.3 Support Python 2.7 framework-style distributions on macOS 12 Apr 2, 2022
@nickhutchinson
Copy link
Contributor Author

I want to try and test in a macOS 12.2 VM -- rereading the original thread more carefully, it might be that the behaviour change was actually in macOS 12.0, and Apple's Python 2 (removed in macOS 12.3) had some custom patches to work around the issue.

This does seem to be the case, and I've updated the PR description and commit message to be more accurate. Virtualenv worked with Apple's Python 2.7 on macOS [12,12.3) -- though not with Python 2.7 distributions from python.org -- likely because they applied a bunch of custom patches, including this one: https://github.com/apple-oss-distributions/python/blob/python-170.80.2/2.7/fix/getpath.c.ed.

@hynek
Copy link

hynek commented Apr 7, 2022

I want to try and test in a macOS 12.2 VM -- rereading the original thread more carefully, it might be that the behaviour change was actually in macOS 12.0, and Apple's Python 2 (removed in macOS 12.3) had some custom patches to work around the issue.

This is 100% the case; I remember switching to Apple's 2.7 because "suddenly" virtualenv was broken and forgetting to further investigate.

Copy link
Contributor

@gaborbernat gaborbernat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nickhutchinson can you make the CI pass? thanks!

virtualenv's trick of adding a symlink to the base Python dylib
(`/Library/Frameworks/Python.framework/Versions/2.7/Python`) in the
virtualenv root, and rewriting the dependent dylib list of the
`<virtualenv>/bin/python` binary with the path of this symlink, no
longer works in macOS 12.

As of macOS 12, when <https://github.com/python/cpython/blob/v2.7.18/Modules/getpath.c>
computes `sys.prefix`, it ends up resolving the symlink. The result is
that `sys.prefix` points to the Python framework, e.g.
`/path/to/Python.framework/Versions/2.7` instead of at the virtualenv
root.

To mitigate this, add a _copy_ of the Python dylib into the virtualenv
instead of a symlink. To placate code signing (at least on Intel
machines), add a symlink to the `Resources` dir so the `Info.plist` file
is accessible.

Note that Apple's Python 2.7 distribution -- removed in macOS 12.3 but
previously located at /System/Library/Frameworks/Python.framework --
behaved differently to the python.org builds. It appears to have
incorporated custom patches to getpath.c to support virtualenv. See:
<https://github.com/apple-oss-distributions/python/blob/python-170.80.2/2.7/fix/getpath.c.ed>.
@gaborbernat gaborbernat merged commit 63e80ff into pypa:main Apr 11, 2022
@evangrim
Copy link

evangrim commented Sep 20, 2022

@hynek You mentioned switching to Apple's 2.7. How did you manage that? I'm asking as an end user of an old virtualenv project, and I understand little about virtualenv or config related to it.z

Did switching help you on an older version of virtualenv, or only on an updated version?

@hynek
Copy link

hynek commented Sep 21, 2022

@evangrim What I meant was that I didn't install Python 2 myself anymore & used the one that came with macOS. The point is moot tho, because Apple doesn't ship a Python 2 anymore.

Nowadays, the pyenv version works just fine with current virtualenv releases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Virtualenv does not work correctly on python 2.7 from python.org and new macOS 12 Monterey
4 participants