Skip to content

python3Minimal: Add top-level for a minimal Python 3 build #66762

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
flokli merged 8 commits into from
Aug 21, 2019
Merged

python3Minimal: Add top-level for a minimal Python 3 build #66762

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
5c0d493
python: Enable building without openssl
adisbladis Aug 17, 2019
414cde4
python: allow stripping IDLE
flokli Aug 18, 2019
d03283c
python: allow stripping tests
flokli Aug 18, 2019
8335f3d
python: allow stripping config
flokli Aug 18, 2019
52fcbbf
python: make rebuilding bytecode optional
flokli Aug 18, 2019
db087f2
python: allow stripping bytecode
flokli Aug 18, 2019
100eae3
python: allow stripping tkinter
flokli Aug 18, 2019
99e6ca2
python3Minimal: Add top-level for a minimal Python 3 build
adisbladis Aug 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 24 additions & 3 deletions pkgs/development/interpreters/python/cpython/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@
, sha256
, passthruFun
, bash
, stripConfig ? false
, stripIdlelib ? false
, stripTests ? false
, stripTkinter ? false
, rebuildBytecode ? true
, stripBytecode ? false
}:

assert x11Support -> tcl != null
Expand Down Expand Up @@ -134,6 +140,7 @@ in with passthru; stdenv.mkDerivation {
"--without-ensurepip"
"--with-system-expat"
"--with-system-ffi"
] ++ optionals (openssl != null) [
"--with-openssl=${openssl.dev}"
] ++ optionals (stdenv.hostPlatform != stdenv.buildPlatform) [
"ac_cv_buggy_getaddrinfo=no"
Expand Down Expand Up @@ -221,8 +228,20 @@ in with passthru; stdenv.mkDerivation {
find $out/lib/python*/config-* -type f -print -exec nuke-refs -e $out '{}' +
find $out/lib -name '_sysconfigdata*.py*' -print -exec nuke-refs -e $out '{}' +

'' + optionalString stripConfig ''
rm -R $out/bin/python*-config $out/lib/python*/config-*
'' + optionalString stripIdlelib ''
# Strip IDLE (and turtledemo, which uses it)
rm -R $out/bin/idle* $out/lib/python*/{idlelib,turtledemo}
'' + optionalString stripTkinter ''
rm -R $out/lib/python*/tkinter
'' + optionalString stripTests ''
# Strip tests
rm -R $out/lib/python*/test $out/lib/python*/**/test{,s}
'' + ''
# Include a sitecustomize.py file
cp ${../sitecustomize.py} $out/${sitePackages}/sitecustomize.py
'' + optionalString rebuildBytecode ''

# Determinism: rebuild all bytecode
# We exclude lib2to3 because that's Python 2 code which fails
Expand All @@ -232,6 +251,8 @@ in with passthru; stdenv.mkDerivation {
find $out -name "*.py" | ${pythonForBuildInterpreter} -m compileall -q -f -x "lib2to3" -i -
find $out -name "*.py" | ${pythonForBuildInterpreter} -O -m compileall -q -f -x "lib2to3" -i -
find $out -name "*.py" | ${pythonForBuildInterpreter} -OO -m compileall -q -f -x "lib2to3" -i -
'' + optionalString stripBytecode ''
find $out -type d -name __pycache__ -print0 | xargs -0 -I {} rm -rf "{}"
'';

preFixup = stdenv.lib.optionalString (stdenv.hostPlatform != stdenv.buildPlatform) ''
Expand All @@ -241,9 +262,9 @@ in with passthru; stdenv.mkDerivation {

# Enforce that we don't have references to the OpenSSL -dev package, which we
# explicitly specify in our configure flags above.
disallowedReferences = [
openssl.dev
] ++ stdenv.lib.optionals (stdenv.hostPlatform != stdenv.buildPlatform) [
disallowedReferences =
stdenv.lib.optionals (openssl != null) [ openssl.dev ]
++ stdenv.lib.optionals (stdenv.hostPlatform != stdenv.buildPlatform) [
# Ensure we don't have references to build-time packages.
# These typically end up in shebangs.
pythonForBuild buildPackages.bash
Expand Down
32 changes: 32 additions & 0 deletions pkgs/development/interpreters/python/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,38 @@ in {
inherit passthruFun;
};

# Minimal versions of Python (built without optional dependencies)
python3Minimal = (callPackage ./cpython {
self = python3Minimal;
sourceVersion = {
major = "3";
minor = "7";
patch = "4";
suffix = "";
};
sha256 = "0gxiv5617zd7dnqm5k9r4q2188lk327nf9jznwq9j6b8p0s92ygv";
inherit (darwin) CF configd;
inherit passthruFun;

# strip down that python version as much as possible
openssl = null;
readline = null;
ncurses = null;
gdbm = null;
sqlite = null;
stripConfig = true;
Copy link
Copy Markdown
Member

@Mic92 Mic92 Aug 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this prevent building c extensions or embed this version of python in some projects depending on python-config?

Copy link
Copy Markdown
Member

@flokli flokli Aug 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. python-config from the passed python would be used to get include and linker flags, and as it doesn't exist, you won't be able to use it.

Copy link
Copy Markdown
Member

@flokli flokli Aug 21, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sidenote: It seems you only need python-config if you want to reuse that version of python in another project, but building a python3Minimal.withPackages calling some C bindings works just fine.

stripIdlelib = true;
stripTests = true;
stripTkinter = true;
rebuildBytecode = false;
stripBytecode = true;
}).overrideAttrs(old: {
pname = "python3-minimal";
meta = old.meta // {
maintainers = [];
};
});

pypy27 = callPackage ./pypy {
self = pypy27;
sourceVersion = {
Expand Down
2 changes: 1 addition & 1 deletion pkgs/top-level/all-packages.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8754,7 +8754,7 @@ in
python3Packages = python3.pkgs;

pythonInterpreters = callPackage ./../development/interpreters/python {};
inherit (pythonInterpreters) python27 python35 python36 python37 python38 pypy27 pypy36;
inherit (pythonInterpreters) python27 python35 python36 python37 python38 python3Minimal pypy27 pypy36;

# Python package sets.
python27Packages = lib.hiPrioSet (recurseIntoAttrs python27.pkgs);
Expand Down