Skip to content

WIP: Add garbage collector to docker registry #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ironpinguin wants to merge 11 commits into from
Closed

WIP: Add garbage collector to docker registry #1

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
ca7d236
Add garbage collector to docker registry
ironpinguin Apr 6, 2018
179ef9a
Extend Test with run gc
ironpinguin Apr 6, 2018
17d348a
change to mkEnableOption and fix typos
ironpinguin Apr 7, 2018
db9f58b
Fix copy & paste string
ironpinguin Apr 7, 2018
ad71bb4
Fix a other typo
ironpinguin Apr 7, 2018
6a8194f
fix evaluation + applying minor fixes
Ma27 Apr 7, 2018
7c31c36
start fixing tests
Ma27 Apr 8, 2018
fa65288
stupid me
Ma27 Apr 8, 2018
ef2fd10
review comment
Ma27 Apr 8, 2018
8d05940
just restart in the end
Ma27 Apr 8, 2018
384749a
Add wait for open docker-registry port
Apr 8, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 37 additions & 3 deletions nixos/modules/services/misc/docker-registry.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ let
};
};

configFile = pkgs.writeText "docker-registry-config.yml" (builtins.toJSON (registryConfig // cfg.extraConfig));

in {
options.services.dockerRegistry = {
enable = mkEnableOption "Docker Registry";
Expand Down Expand Up @@ -95,16 +97,32 @@ in {
default = {};
type = types.attrsOf types.str;
};

enableGarbageCollect = mkOption {

@Ma27 Ma27 Apr 6, 2018

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you might want to have a look at mkEnableOption

description = "Enable run GarbageCollect automatic every given time.";
default = false;
type = types.bool;
};

garbageCollectDates = mkOption {
default = "daily";
type = types.str;
description = ''
Specification (in the format described by
<citerefentry><refentrytitle>systemd.time</refentrytitle>
<manvolnum>7</manvolnum></citerefentry>) of the time at
which the garbage collect will occur.
''
};

@Ma27 Ma27 Apr 6, 2018

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the indent and the additional closing bracket are obviously broken %)

};
};

config = mkIf cfg.enable {
systemd.services.docker-registry = {
description = "Docker Container Registry";
wantedBy = [ "multi-user.target" ];
after = [ "network.target" ];
script = let
configFile = pkgs.writeText "docker-registry-config.yml" (builtins.toJSON (registryConfig // cfg.extraConfig));
in ''
script = ''
${pkgs.docker-distribution}/bin/registry serve ${configFile}
'';

Expand All @@ -114,6 +132,22 @@ in {
};
};

systemd.services.docker-registry-garbage-collect = {
description = "Run Garbage Collection for docker registry";

restartIfChange = false;
unitConfig.X-StopOnRemoval = false;

serviceConfig.Type = "oneshot";

script = ''
${pkgs.docker-distribution}/bin/registry garbage-collect ${configFile}
${lib.optionalString enableRedisCache '${pkgs.systemd}/bin/systemctl restart docker-registry'}

@Ma27 Ma27 Apr 6, 2018

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

single quotes won't work as single quotes can be part of variable names.
Use '' for multiline strings or " for single line strings and you're safe %)

'';

startAt = optional cfg.enableGarbageCollect cfg.garbageCollectDates;
};

users.extraUsers.docker-registry = {
createHome = true;
home = cfg.storagePath;
Expand Down
12 changes: 10 additions & 2 deletions nixos/tests/docker-registry.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import ./make-test.nix ({ pkgs, ...} : {
name = "docker-registry";
meta = with pkgs.stdenv.lib.maintainers; {
maintainers = [ globin ma27 ];
maintainers = [ globin ma27 ironpinguin ];
};

nodes = {
Expand Down Expand Up @@ -43,7 +43,15 @@ import ./make-test.nix ({ pkgs, ...} : {
$client2->succeed("docker images | grep scratch");

$client2->succeed(
'curl -fsS -X DELETE registry:8080/v2/scratch/manifests/$(curl registry:8080/v2/scratch/manifests/latest | jq ".fsLayers[0].blobSum" | sed -e \'s/"//g\')'
'curl -fsS -X DELETE registry:8080/v2/scratch/manifests/$(curl -fsS -I -H"Accept: application/vnd.docker.distribution.manifest.v2+json" registry:8080/v2/scratch/manifests/latest | grep Docker-Content | sed -e\'Docker-Content-Digest: //\' | tr -d \'\r\')'

@Ma27 Ma27 Apr 6, 2018

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember exactly how the JSON looked like, however I think that a solution with jq might be way easier to read

@ironpinguin ironpinguin Apr 7, 2018

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. The json looks now different with the Header Accept: application/vnd.docker.distribution.manifest.v2+json
  2. The correct digest is not part of the json but of the HTTP Header. And also it is only the correct one with the given Header :-(

@ironpinguin ironpinguin Apr 7, 2018

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a example to see it better:
curl -fsS -i -H'Accept: application/vnd.docker.distribution.manifest.v2+json' localhost:5000/v2/scratch/manifests/latest

HTTP/1.1 200 OK
Content-Length: 522
Content-Type: application/vnd.docker.distribution.manifest.v2+json
Docker-Content-Digest: sha256:0eec87df34024c2c53db0faae043ab2175902c5877a488eb9053413e03f32661
Docker-Distribution-Api-Version: registry/2.0
Etag: "sha256:0eec87df34024c2c53db0faae043ab2175902c5877a488eb9053413e03f32661"
X-Content-Type-Options: nosniff
Date: Sat, 07 Apr 2018 09:36:57 GMT

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 887,
      "digest": "sha256:b8edfce180eb3b092c8248c81f8c208b6f835308a2041d11cd2389d6741035ad"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 49,
         "digest": "sha256:3c2cba919283a210665e480bcbf943eaaf4ed87a83f02e81bb286b8bdead0e75"
      }
   ]
}

All 3 digests are also blobs:

  • sha256:0eec87df34024c2c53db0faae043ab2175902c5877a488eb9053413e03f32661 contains the images meta information
  • sha256:b8edfce180eb3b092c8248c81f8c208b6f835308a2041d11cd2389d6741035ad
    contains the container config as json
  • sha256:3c2cba919283a210665e480bcbf943eaaf4ed87a83f02e81bb286b8bdead0e75 is the first root fs layer referenced by both other blobs json's

);

$registry->succeed("systemctl start docker-registry-garbage-collect");
$registry->waitForUnit("docker-registry.service");

$registry->fail("ls store/docker/registry/v2/blobs/sha256/**/data");

$client1->succeed("docker push registry:8080/scratch");
$registry->succeed("ls store/docker/registry/v2/blobs/sha256/**/data");
'';
})