python-env: wrap python executable with --resolve-argv0#462090
python-env: wrap python executable with --resolve-argv0#462090mweinelt merged 1 commit intoNixOS:stagingfrom
Conversation
nixpkgs-ci
bot
added
10.rebuild-linux: 501+
There was a problem hiding this comment.
Thanks a lot! I figured it was something in the wrapper, but didn’t dig into it yet.
Could you please add the justification for this to the commit message, so future contributors can see what happened here?
(Furthermore, wondering out loud: why is this only an issue on Darwin?!)
|
I am rebuilding the whole darwin world to see if this pr does fix #461406. |
This commit, together with NixOS#442540, changes the way python environments are built: * When generating wrappers for python executables, we inherit argv[0] from the wrapper. This causes python to initialize its configuration in the environment with all the correct paths. * We also resolve argv[0] to absolute path when invoking python from PATH. This helps set python's prefix correctly on Darwin. The end result is that python environments no longer appear to be venvs, and behave more like a vanilla python installation. In addition it's possible to create a venv using an environment and use packages from both the environment and the venv.
|
tested build fish on local python-venv branch. wait for nixpkgs-review. |
That’s going to take a while 😓 |
|
|
nixpkgs-review failed due to non-reproducible failure of pytest-xdist (will likely to fail with cores > 1) |
|
qbisi
added
the
0.kind: ZHF Fixes
|
Via staging and staging-25.11. |
|
Successfully created backport PR for |
github-actions
bot
added
the
8.has: port to stable
Investigated removing the nixpkgs pin now that fish 4.2.1 test failures are resolved (PR #462589 disabled checks on darwin as temporary fix while PR #462090's Python wrapper fixes go through staging). However, latest nixpkgs-unstable (Nov 20) has a new issue: pyarrow build failures on darwin due to multiple conflicting protobuf versions (30, 31, 33) in the dependency closure causing memory corruption. Issue: NixOS/nixpkgs#461396 Pending fixes: PRs #461569, #461572 (still in staging, not merged) Keeping the Nov 8 pin until the protobuf fixes land in nixpkgs-unstable. The fish issue won't affect builds since checks are disabled on darwin. Related: NixOS/nixpkgs#461406 (fish), NixOS/nixpkgs#462090, NixOS/nixpkgs#462589
Investigated removing the nixpkgs pin now that fish 4.2.1 test failures are resolved (PR #462589 disabled checks on darwin as temporary fix while PR #462090's Python wrapper fixes go through staging). However, latest nixpkgs-unstable (Nov 20) has a new issue: pyarrow build failures on darwin due to multiple conflicting protobuf versions (30, 31, 33) in the dependency closure causing memory corruption. Issue: NixOS/nixpkgs#461396 Pending fixes: PRs #461569, #461572 (still in staging, not merged) Keeping the Nov 8 pin until the protobuf fixes land in nixpkgs-unstable. The fish issue won't affect builds since checks are disabled on darwin. Related: NixOS/nixpkgs#461406 (fish), NixOS/nixpkgs#462090, NixOS/nixpkgs#462589
|
FYI workaround: # https://github.com/NixOS/nixpkgs/pull/462090
horribleHackToFixPythonWithPackages =
ppkgs:
ppkgs.overrideAttrs (oldAttrs: {
postBuild =
assert !(lib.hasInfix "--resolve-argv0" oldAttrs.postBuild);
lib.replaceString "--inherit-argv0" "--inherit-argv0 --resolve-argv0" oldAttrs.postBuild;
}); |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
|
https://nixpkgs-tracker.ocfox.me/?pr=462090, #461884 (comment) and #471587 (I can't access nixpk.gs/pr-tracker) |
6 participants
original pr: #442540
"--resolve-argv0" resolve python's argv[0] to absolute path on darwin platform. This help setting python's prefix correctly on darwin platform when invoking python from PATH. (note on linux platform we dont need "--resolve-argv0")
should fix #461406
Things done
build legacyPackages.aarch64-darwin.fish
passthru.tests.nixpkgs-reviewon this PR. See nixpkgs-review usage../result/bin/.Add a 👍 reaction to pull requests you find important.