virtualisation/rosetta: Perserve ArgvZero to fix python virtualenv creation#463106
Conversation
…eation Preserving ArgvZero is supported from MacOS 14 Sonoma forward, which is the current minimum supported version of nixpkgs. See https://developer.apple.com/documentation/virtualization/running-intel-binaries-in-linux-vms-with-rosetta#Mount-the-Shared-Directory-and-Register-Rosetta The necessity of this fix came up with python virtual environments, because python uses the path of the executable (argv zero) to determine if it should load a virtual environment or not. In the past this was overriden with NIX_PYTHONEXECUTABLE, NIX_PYTHONPATH and NIX_PYTHONPREFIX, but we stopped doing that for the python interpreter so it works more like a normal python interpreter in nix. Without those variables, in the rosetta emulated case, python was no longer able to determine where it was called from, and thus could not correctly activate virtual environments in that case.
|
|
winterqt
left a comment
There was a problem hiding this comment.
Thanks a ton!
(Nit: I’m not sure if this can be considered virtualenv-specific, as it happens when using python.withPackages too, which I don’t think is a venv. Actual change LGTM, though.)
|
Ah, another thing: could you add a reference to the commit and/or PR that broke this to the commit message, please? Just to save future archeologists a headache 😉 |
Arguably this is fine given the other commits that remove the NIX_* stuff mention venvs. |
|
@winterqt I added a link to the pull request that broke things - does that work for you? Regarding venv: I would argue that |
|
Nix-Review spits this out, not quite sure if this is actually helpful to judge this works:
|
|
Friendly ping @qbisi @9999years @eclairevoyant - could you perhaps be so kind as to take a look at this pull request? I would really like to get this into 25.11, as that would un-break the review workflow for a lot of people who rely on VMs to check linux x86 binaries from darwin. |
|
I'm not a committer nor have a darwin machine, don't have any input here. |
|
@NixOS/darwin-core - is this something you should perhaps take a look at? |
|
This doesn't fix the issue with Python environments for me, but it doesn't hurt either. With let
pkgs = import ./. { };
in
(pkgs.python3.withPackages (pyPkgs: [
pyPkgs.pandas
])).env |
|
Edit: I'm on aarch64-darwin, not x86_64-darwin or Rosetta x86_64-darwin, so maybe that makes sense? |
Expected, this just fixes that issue with Rosetta. |
|
*with Linux Rosetta on aarch64-darwin |
|
Thanks @winterqt, sorry for the confusion! |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/prs-ready-for-review/3032/6088 |
|
Successfully created backport PR for |
|
@SuperSandro2000 Thanks! |
Preserving ArgvZero is supported from MacOS 14 Sonoma forward, which is the current minimum supported version of nixpkgs. See https://developer.apple.com/documentation/virtualization/running-intel-binaries-in-linux-vms-with-rosetta#Mount-the-Shared-Directory-and-Register-Rosetta
The necessity of this fix came up with python virtual environments, because python uses the path of the executable (argv zero) to determine if it should load a virtual environment or not. In the past this was overriden with NIX_PYTHONEXECUTABLE, NIX_PYTHONPATH and NIX_PYTHONPREFIX, but we stopped doing that for the python interpreter so it works more like a normal python interpreter in nix.
Without those variables, in the rosetta emulated case, python was no longer able to determine where it was called from, and thus could not correctly activate virtual environments in that case.
Investigation and this suggestion for the fix happened here. #461884
This is related to, but not the same issue as was observed here: #461406
This problem was introduced here #442540 where virtual env creation was changed so the interpreter was special cased and no longer sets the above mentioned variables.
I'd like to ping some of the people that helped debug this: @qbisi @9999years @eclairevoyant @winterqt
Some open questions: should this be targeting NixOS:staging instead of master like #462090?
Things done
passthru.tests.nixpkgs-reviewon this PR. See nixpkgs-review usage../result/bin/.Add a 👍 reaction to pull requests you find important.