Skip to content

dockerTools.buildImageWithNixDb: simplifications and switch to closure info #49414

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nlewo merged 1 commit into from
Nov 12, 2018
Merged

dockerTools.buildImageWithNixDb: simplifications and switch to closure info #49414

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 6 additions & 29 deletions pkgs/build-support/docker/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
pkgs,
pigz,
nix,
perl,
runCommand,
rsync,
shadow,
Expand All @@ -23,6 +22,7 @@
referencesByPopularity,
writeScript,
writeText,
closureInfo
}:

# WARNING: this API is unstable and may be subject to backwards-incompatible changes in the future.
Expand Down Expand Up @@ -261,19 +261,6 @@ rec {
${text}
'';

nixRegistration = contents: runCommand "nix-registration" {
buildInputs = [ nix perl ];
# For obtaining the closure of `contents'.
exportReferencesGraph =
let contentsList = if builtins.isList contents then contents else [ contents ];
in map (x: [("closure-" + baseNameOf x) x]) contentsList;
}
''
mkdir $out
printRegistration=1 perl ${pkgs.pathsFromGraph} closure-* > $out/db.dump
perl ${pkgs.pathsFromGraph} closure-* > $out/storePaths
'';

# Create $maxLayers worth of Docker Layers, one layer per store path
# unless there are more paths than $maxLayers. In that case, create
# $maxLayers-1 for the most popular layers, and smush the remainaing
Expand Down Expand Up @@ -795,29 +782,19 @@ rec {
# the container.
# Be careful since this doesn't work well with multilayer.
buildImageWithNixDb = args@{ contents ? null, extraCommands ? "", ... }:
buildImage (args // {
let contentsList = if builtins.isList contents then contents else [ contents ];
in buildImage (args // {
extraCommands = ''
echo "Generating the nix database..."
echo "Warning: only the database of the deepest Nix layer is loaded."
echo " If you want to use nix commands in the container, it would"
echo " be better to only have one layer that contains a nix store."
# This requires Nix 1.12 or higher

export NIX_REMOTE=local?root=$PWD
${nix}/bin/nix-store --load-db < ${nixRegistration contents}/db.dump

# We fill the store in order to run the 'verify' command that
# generates hash and size of output paths.
# Note when Nix 1.12 is be the stable one, the database dump
# generated by the exportReferencesGraph function will
# contains sha and size. See
# https://github.com/NixOS/nix/commit/c2b0d8749f7e77afc1c4b3e8dd36b7ee9720af4a
storePaths=$(cat ${nixRegistration contents}/storePaths)
echo "Copying everything to /nix/store (will take a while)..."
cp -prd $storePaths nix/store/
Copy link
Contributor

@dingxiangfei2009 dingxiangfei2009 Oct 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the closure?

Copy link
Member Author

@nlewo nlewo Oct 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The closure is copied by the buildImage function. This copy was needed to do the --check-contents at the layer level. This is no longer required (with Nix 2.0) to initialize correctly the database.

Copy link
Contributor

@dingxiangfei2009 dingxiangfei2009 Oct 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see.

${nix}/bin/nix-store --verify --check-contents
${nix}/bin/nix-store --load-db < ${closureInfo {rootPaths = contentsList;}}/registration

mkdir -p nix/var/nix/gcroots/docker/
for i in ${lib.concatStringsSep " " contents}; do
for i in ${lib.concatStringsSep " " contentsList}; do
ln -s $i nix/var/nix/gcroots/docker/$(basename $i)
done;
'' + extraCommands;
Expand Down