Skip to content

WIP: Refactor fixpoints and overriding in all-packages #11110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Ericson2314 wants to merge 4 commits into from
Closed

WIP: Refactor fixpoints and overriding in all-packages #11110

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
57568a5
This line is extraneous because `helperFunctions` is part of `self`
Ericson2314 Jun 7, 2015
52c13a6
Lift fix-points out from within `pkgFun` to `pkgFun`'s callsites
Ericson2314 Jun 15, 2015
56056f8
Massage fix-points
Ericson2314 Nov 18, 2015
fae1f0d
WIP: Combined `applyGlobalOverrides` and `overridePackages`
Ericson2314 Nov 18, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 24 additions & 5 deletions doc/functions.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xml:id="chap-functions">
xmlns:xlink="http://www.w3.org/1999/xlink"
xml:id="chap-functions">

<title>Functions reference</title>

Expand Down Expand Up @@ -30,14 +30,33 @@
in ...</programlisting>
</para>

<para>
It can be used multiple times in succession to "chain" overrides:

<programlisting>let
pkgs0 = import &lt;nixpkgs&gt; {};
pkgs1 = pkgs0.overridePackages (self: super: {
foo = null;
};
pkgs2 = pkgs1.overridePackages (self: super: {
foo = 1;
bar = super.foo;
baz = self.bar;
};
pkgs3 = pkgs2.overridePackages (self: super: {
bar = super.foo;
};
in assert foo == 1; assert bar == 1; assert baz = 1; ...</programlisting>
</para>

<para>
The resulting <varname>newpkgs</varname> will have the new <varname>foo</varname>
expression, and all other expressions depending on <varname>foo</varname> will also
use the new <varname>foo</varname> expression.
</para>

<para>
The behavior of this function is similar to <link
The behavior of this function is similar to <link
linkend="sec-modify-via-packageOverrides">config.packageOverrides</link>.
</para>

Expand All @@ -49,7 +68,7 @@ in ...</programlisting>

<para>
The <varname>super</varname> parameter refers to the old package set.
It's equivalent to <varname>pkgs</varname> in the above example.
It's equivalent to <varname>pkgs</varname> in the first example.
</para>

</section>
Expand Down Expand Up @@ -91,7 +110,7 @@ in ...</programlisting>
<para>
The function <varname>overrideDerivation</varname> is usually available for all the
derivations in the nixpkgs expression (<varname>pkgs</varname>).
</para>
</para>
<para>
It is used to create a new derivation by overriding the attributes of
the original derivation according to the given function.
Expand Down
177 changes: 88 additions & 89 deletions pkgs/top-level/all-packages.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,35 +98,102 @@ let
# (un-overriden) set of packages, allowing packageOverrides
# attributes to refer to the original attributes (e.g. "foo =
# ... pkgs.foo ...").
pkgs = applyGlobalOverrides (config.packageOverrides or (pkgs: {}));
pkgs = let
# Absolutely un-overriden packages, used to define stdenv before pkg overrides
pkgsVanilla = pkgsFun pkgsVanilla;
in overridePackages
pkgsVanilla
pkgsFun
# TODO: break compat and use self and super in config too.
(self: super: (config.packageOverrides or (pkgs: {})) super);

mkOverrides = pkgsOrig: overrides: overrides //
(lib.optionalAttrs (pkgsOrig.stdenv ? overrides && crossSystem == null) (pkgsOrig.stdenv.overrides pkgsOrig));

# Return the complete set of packages, after applying the overrides
# returned by the `overrider' function (see above). Warning: this
# function is very expensive!
applyGlobalOverrides = overrider:
# Easily override this package set.
# Warning: this function is very expensive and must not be used
# from within the nixpkgs repository.
#
# See manual / XML for more documentation.
overridePackages = pkgsForStdenv: # pkgSet
oldOverriders: # pkgSet -> pkgSet
overrider: # pkgSet -> pkgSet -> pkgSet
let
# Call the overrider function. We don't want stdenv overrides
# in the case of cross-building, or otherwise the basic
# overrided packages will not be built with the crossStdenv
# adapter.
overrides = mkOverrides pkgsOrig (overrider pkgsOrig);
rawOverrides = overrider self super;

# We don't want stdenv overrides in the case of cross-building, or
# otherwise the basic overrided packages will not be built with the
# crossStdenv adapter.
overrides = rawOverrides // (lib.optionalAttrs
(pkgsForStdenv.stdenv ? overrides && crossSystem == null)
(pkgsForStdenv.stdenv.overrides pkgsForStdenv));

# The un-overriden packages, passed to `overrider'.
pkgsOrig = pkgsFun pkgs {};
# The shallowly un-overriden packages, passed to `overrider'.
super = oldOverriders self;

# Allow downstream overriding
overrideMore = nextOverrider: overridePackages
self
(self: ((overrider self) (oldOverriders self)))
nextOverrider;

# The overriden, final packages.
pkgs = pkgsFun pkgs overrides;
in pkgs;
self = super // overrides // { overridePackages = overrideMore; };
in self;


# The package compositions. Yes, this isn't properly indented.
pkgsFun = pkgs: overrides:
with helperFunctions;
let defaultScope = pkgs // pkgs.xorg; self = self_ // overrides;
self_ = with self; helperFunctions // {
pkgsFun = pkgs: let
defaultScope = pkgs // pkgs.xorg;

rawAliases = with pkgs; rec {
adobeReader = adobe-reader;
arduino_core = arduino-core; # added 2015-02-04
asciidocFull = asciidoc-full; # added 2014-06-22
bridge_utils = bridge-utils; # added 2015-02-20
buildbotSlave = buildbot-slave; # added 2014-12-09
cheetahTemplate = pythonPackages.cheetah; # 2015-06-15
clangAnalyzer = clang-analyzer; # added 2015-02-20
cool-old-term = cool-retro-term; # added 2015-01-31
cv = progress; # added 2015-09-06
enblendenfuse = enblend-enfuse; # 2015-09-30
exfat-utils = exfat; # 2015-09-11
firefoxWrapper = firefox-wrapper;
fuse_exfat = exfat; # 2015-09-11
haskell-ng = haskell; # 2015-04-19
haskellngPackages = haskellPackages; # 2015-04-19
htmlTidy = html-tidy; # added 2014-12-06
inherit (haskell.compiler) jhc uhc; # 2015-05-15
inotifyTools = inotify-tools;
jquery_ui = jquery-ui; # added 2014-09-07
libtidy = html-tidy; # added 2014-12-21
lttngTools = lttng-tools; # added 2014-07-31
lttngUst = lttng-ust; # added 2014-07-31
nfsUtils = nfs-utils; # added 2014-12-06
quassel_qt5 = kf5Packages.quassel_qt5; # added 2015-09-30
quasselClient_qt5 = kf5Packages.quasselClient_qt5; # added 2015-09-30
quasselDaemon_qt5 = kf5Packages.quasselDaemon; # added 2015-09-30
quassel_kf5 = kf5Packages.quassel; # added 2015-09-30
quasselClient_kf5 = kf5Packages.quasselClient; # added 2015-09-30
rdiff_backup = rdiff-backup; # added 2014-11-23
rssglx = rss-glx; #added 2015-03-25
rxvt_unicode_with-plugins = rxvt_unicode-with-plugins; # added 2015-04-02
speedtest_cli = speedtest-cli; # added 2015-02-17
sqliteInteractive = sqlite-interactive; # added 2014-12-06
x11 = xlibsWrapper; # added 2015-09
xf86_video_nouveau = xorg.xf86videonouveau; # added 2015-09
xlibs = xorg; # added 2015-09
youtube-dl = pythonPackages.youtube-dl; # added 2015-06-07
youtubeDL = youtube-dl; # added 2014-10-26
pidginlatexSF = pidginlatex; # added 2014-11-02
};

tweakAlias = _n: alias: with lib;
if alias.recurseForDerivations or false
then removeAttrs alias ["recurseForDerivations"]
else alias;

aliases = lib.mapAttrs tweakAlias rawAliases;

in with pkgs; helperFunctions // aliases // {

# Make some arguments passed to all-packages.nix available
inherit system platform;
Expand All @@ -145,23 +212,6 @@ let

newScope = extra: lib.callPackageWith (defaultScope // extra);

# Easily override this package set.
# Warning: this function is very expensive and must not be used
# from within the nixpkgs repository.
#
# Example:
# pkgs.overridePackages (self: super: {
# foo = super.foo.override { ... };
# }
#
# The result is `pkgs' where all the derivations depending on `foo'
# will use the new version.
overridePackages = f:
let
newpkgs = pkgsFun newpkgs overrides;
overrides = mkOverrides pkgs (f newpkgs pkgs);
in newpkgs;

# Override system. This is useful to build i686 packages on x86_64-linux.
forceSystem = system: kernel: (import ./all-packages.nix) {
inherit system;
Expand Down Expand Up @@ -15350,56 +15400,5 @@ let

mg = callPackage ../applications/editors/mg { };

}; # self_ =


### Deprecated aliases - for backward compatibility

aliases = with self; rec {
adobeReader = adobe-reader;
arduino_core = arduino-core; # added 2015-02-04
asciidocFull = asciidoc-full; # added 2014-06-22
bridge_utils = bridge-utils; # added 2015-02-20
buildbotSlave = buildbot-slave; # added 2014-12-09
cheetahTemplate = pythonPackages.cheetah; # 2015-06-15
clangAnalyzer = clang-analyzer; # added 2015-02-20
cool-old-term = cool-retro-term; # added 2015-01-31
cv = progress; # added 2015-09-06
enblendenfuse = enblend-enfuse; # 2015-09-30
exfat-utils = exfat; # 2015-09-11
firefoxWrapper = firefox-wrapper;
fuse_exfat = exfat; # 2015-09-11
haskell-ng = haskell; # 2015-04-19
haskellngPackages = haskellPackages; # 2015-04-19
htmlTidy = html-tidy; # added 2014-12-06
inherit (haskell.compiler) jhc uhc; # 2015-05-15
inotifyTools = inotify-tools;
jquery_ui = jquery-ui; # added 2014-09-07
libtidy = html-tidy; # added 2014-12-21
lttngTools = lttng-tools; # added 2014-07-31
lttngUst = lttng-ust; # added 2014-07-31
nfsUtils = nfs-utils; # added 2014-12-06
quassel_qt5 = kf5Packages.quassel_qt5; # added 2015-09-30
quasselClient_qt5 = kf5Packages.quasselClient_qt5; # added 2015-09-30
quasselDaemon_qt5 = kf5Packages.quasselDaemon; # added 2015-09-30
quassel_kf5 = kf5Packages.quassel; # added 2015-09-30
quasselClient_kf5 = kf5Packages.quasselClient; # added 2015-09-30
rdiff_backup = rdiff-backup; # added 2014-11-23
rssglx = rss-glx; #added 2015-03-25
rxvt_unicode_with-plugins = rxvt_unicode-with-plugins; # added 2015-04-02
speedtest_cli = speedtest-cli; # added 2015-02-17
sqliteInteractive = sqlite-interactive; # added 2014-12-06
x11 = xlibsWrapper; # added 2015-09
xf86_video_nouveau = xorg.xf86videonouveau; # added 2015-09
xlibs = xorg; # added 2015-09
youtube-dl = pythonPackages.youtube-dl; # added 2015-06-07
youtubeDL = youtube-dl; # added 2014-10-26
pidginlatexSF = pidginlatex; # added 2014-11-02
};

tweakAlias = _n: alias: with lib;
if alias.recurseForDerivations or false then
removeAttrs alias ["recurseForDerivations"]
else alias;

in lib.mapAttrs tweakAlias aliases // self; in pkgs
in pkgs