annoying errors when image tags are removed

[glensc]

so, in the past i have had images named ed/php/7.0-fpm:latest, but if deleted the latest tag or whole 7.0-fpm image, i get the error.

i've tried two scenarios:

• deleting latest in gitlab ui (to delete tag)
• deleting ed/php/7.0-fpm in gitlab (to delete image and all tags)

the result is the same. the error below.

i've "fixed" this by pushing dummy image under ed/php/7.0-fpm to keep the script going without crashing.

$ docker-compose run --rm gitlab-registry-cleanup
Creating a Pipfile for this project…
Traceback (most recent call last):
  File "/usr/local/bin/gitlab-registry-cleanup", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/gitlab_registry_cleanup/cli.py", line 204, in main
    cleanup_gitlab_registry(args.gitlab_server, args.registry_server, args.username, args.password, args.dry_run)
  File "/usr/local/lib/python3.6/site-packages/gitlab_registry_cleanup/cli.py", line 195, in cleanup_gitlab_registry
    gitlab_base_url, registry_base_url, username, password, dry_run=dry_run, notify_callback=console_output
  File "/usr/local/lib/python3.6/site-packages/gitlab_registry_cleanup/cleanup.py", line 19, in soft_delete_untagged_imagehashes
    untagged_imagehashes = local_registry.repository_untagged_imagehashes[repository]
  File "/usr/local/lib/python3.6/site-packages/gitlab_registry_cleanup/registry.py", line 120, in repository_untagged_imagehashes
    self._repository_untagged_imagehashes = self._find_repository_untagged_imagehashes()
  File "/usr/local/lib/python3.6/site-packages/gitlab_registry_cleanup/registry.py", line 89, in _find_repository_untagged_imagehashes
    for repository_path in self.repository_paths
  File "/usr/local/lib/python3.6/site-packages/gitlab_registry_cleanup/registry.py", line 89, in <dictcomp>
    for repository_path in self.repository_paths
  File "/usr/local/lib/python3.6/site-packages/gitlab_registry_cleanup/registry.py", line 114, in repository_tagged_imagehashes
    self._repository_tagged_imagehashes = self._find_repository_tagged_imagehashes()
  File "/usr/local/lib/python3.6/site-packages/gitlab_registry_cleanup/registry.py", line 68, in _find_repository_tagged_imagehashes
    for tag in os.listdir(os.path.join(self._docker_repositories_root, repository_path, TAGS_DIRECTORY)):
FileNotFoundError: [Errno 2] No such file or directory: '/var/opt/gitlab/gitlab-rails/shared/registry/docker/registry/v2/repositories/ed/php/7.0-fpm/_manifests/tags'

[glensc]

here's similar problem. with some filenames:

so, _manifests/tags is missing but _manifests/revisions/sha256 has dirs.

could be gitlab bug, as my intention was to delete the image/tags

# docker-compose run --rm gitlab-registry-cleanup  bash -l
root@3375e2085684:/app# ls -l /var/opt/gitlab/gitlab-rails/shared/registry/docker/registry/v2/repositories/ed/kubectl/dev/_manifests
total 0
drwxr-xr-x 3 root root 20 May 18 20:32 revisions
root@3375e2085684:/app# ls -l /var/opt/gitlab/gitlab-rails/shared/registry/docker/registry/v2/repositories/ed/kubectl/dev/_manifests/revisions/
total 4
drwxr-xr-x 18 root root 4096 Jun  6 20:35 sha256
root@3375e2085684:/app# ls -l /var/opt/gitlab/gitlab-rails/shared/registry/docker/registry/v2/repositories/ed/kubectl/dev/_manifests/revisions/sha256/
total 0
drwxr-xr-x 2 root root 18 May 18 22:57 0df188752ceb14eb89fcf4ed5b77122f1ffbe44f3c74ea4f9fa39537b56070a6
drwxr-xr-x 2 root root 18 May 18 23:35 14a0f9fcf238cb2928f93ef20f960e9adac2a7a27819ea1012627e269c24b28b
drwxr-xr-x 2 root root 18 May 18 23:27 1dab860d455a2feae7c634c10a696b57ba19bcf802d4344ab5dcafe490c25a83
drwxr-xr-x 2 root root 18 May 19 02:20 223130c2ef412d37084dc5bb34b767e70feda90ef15d2ac3a3a80942f3b8f87e
drwxr-xr-x 2 root root 18 May 19 14:20 26e21361c0895322eff25b4c6d7a537b0fafb790e92d1e5a6cee0d89b9759181
drwxr-xr-x 2 root root 18 May 19 00:29 530cf68d8337efbd07b29a7b1a0d54a1b9f86339b720ab7c1305901fc508507b
drwxr-xr-x 2 root root 18 May 18 23:15 55cd204a82a02d452c712b2e7221e48053e4170775f26dbd0760b8a0e0fb02da
drwxr-xr-x 2 root root 18 May 18 20:32 7565f2939160ef2e8e68f687f51fc84eb72bd05e93318a2d70e08c57dac210c6
drwxr-xr-x 2 root root 18 May 19 02:25 b1f493cdaae59358241d8870d2796b3d02f06e3d97a495772779fabe5a1cd0a9
drwxr-xr-x 2 root root 18 May 18 23:10 c3c4a6e3fea787d8f56ae599077e0145875f058c0860e4751763ae7a1c758a29
drwxr-xr-x 2 root root 18 May 19 00:13 ca2110a51cfa27ac735d22791367106088000c144a77e6297c76ca87cdee4904
drwxr-xr-x 2 root root 18 May 18 21:58 d8646c3bf9f7967cf5f691fcf7203b361138bdb045184dc44df8bde956eb11c7
drwxr-xr-x 2 root root 18 May 19 17:36 e4f80d56d4dd5740a1582250b473e74722b6dbe48d297a9a8b74e3715d35d10f
drwxr-xr-x 2 root root 18 May 18 20:34 e6ac671162c651e6300bb2f249b45ab594fdaa84ec9b6cce9a2a5f950d74068e
drwxr-xr-x 2 root root 18 May 19 01:48 f0fca07cbbba955e1680629b99c01dfa3fb7726d02953b4540cf191f96e14af4
drwxr-xr-x 2 root root 18 May 18 21:25 f55d2e3c5568c126db848b1f7d4ec4f5df8ef8650f9cbfc512150c81fb494c80
root@3375e2085684:/app#

[IngoMeyer441]

Could you try the new commit in the develop branch?
GitHub closed this issue automatically after pushing the fix. Feel free to reopen if there is still a problem.

[glensc]

perhaps release new package version? or how do i install development version with pipenv?

[glensc]

tried this and that. failed. asked for help: pypa/pipenv#2685

tried:

--- a/gitlab-registry-cleanup/Pipfile
+++ b/gitlab-registry-cleanup/Pipfile
@@ -4,7 +4,7 @@ verify_ssl = true
 name = "pypi"

 [packages]
-gitlab-registry-cleanup = "*"
+gitlab-registry-cleanup = {git = "https://github.com/sciapp/gitlab-registry-cleanup", editable = true, ref="develop"}

and end up with:

pkg_resources.DistributionNotFound: The 'gitlab-registry-cleanup' distribution was not found and is required by the application

[glensc]

ok. it was error on my side. i had bind mounted wrong directory under docker compose.

your patch seems working. but Python crashed with Segmentation fault. probably not related to this?

[IngoMeyer441]

Hmm, a segmentation fault is strange because this project is pure Python and it is unlikely that Python itself caused it. How do you invoke the gitlab-registry-cleanup command?

[glensc]

yes. Segfault is likely unrelated. and haven't seen it anymore.

not sure what you mean by your "how question". i ran it docker container installed with pipenv.

[IngoMeyer441]

pipenv with Docker should be ok. We can reopen the issue if the segmentation fault will happen again.

[glensc]

please tag new version in pip index. the fix works fine.

[IngoMeyer441]

Done! 😃

[glensc]

update: you released as 0.1.2

which can be installed as:

pipenv install 'gitlab-registry-cleanup>=0.1.2'