Unable to build using existing opensuse-tumbleweed-x86_64 chroot cache

[kilesduli]

Short description of the problem

The first time I run mock -r opensuse-tumbleweed-x86_64 package.src.rpm, it builds just fine. But when I try to build again, it hits package installation errors and can't continue.

INFO: Buildroot is handled by package management downloaded with a bootstrap image:
  rpm-4.19.1.1-3.4.x86_64
  python3-dnf-4.18.0-6.1.noarch
  python3-dnf-plugins-core-4.4.4-2.1.noarch
Start: dnf4 update
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
openSUSE Tumbleweed - x86_64 - OSS                                                                                             7.5 kB/s |  10 kB     00:01    
Error: 
 Problem: problem with installed package systemd-presets-branding-MicroOS-20230214-3.2.noarch
  - cannot install the best update candidate for package systemd-presets-branding-MicroOS-20230214-3.2.noarch
  - both package systemd-presets-branding-Kalpa-20231005-1.4.noarch from opensuse-tumbleweed-oss and systemd-presets-branding-Aeon-20231005-5.1.noarch from opensuse-tumbleweed-oss obsolete systemd-presets-branding-MicroOS < 20231005
  - both package systemd-presets-branding-Aeon-20231005-5.1.noarch from opensuse-tumbleweed-oss and systemd-presets-branding-Kalpa-20231005-1.4.noarch from opensuse-tumbleweed-oss obsolete systemd-presets-branding-MicroOS < 20231005
  - package systemd-presets-branding-Aeon-20231005-5.1.noarch from opensuse-tumbleweed-oss conflicts with systemd-presets-branding provided by systemd-presets-branding-Kalpa-20231005-1.4.noarch from opensuse-tumbleweed-oss
  - package systemd-presets-branding-Kalpa-20231005-1.4.noarch from opensuse-tumbleweed-oss conflicts with systemd-presets-branding provided by systemd-presets-branding-Aeon-20231005-5.1.noarch from opensuse-tumbleweed-oss
(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
WARNING: DNF4 command failed, retrying, attempt #2, sleeping 20s

Output of rpm -q mock

mock-5.9-1.fc41.noarch

Steps to reproduce issue

1. Download caddy-2.8.4-1.suse.tw.src.rpm(from copr)
2. mock -r opensuse-tumbleweed-x86_64 --init
3. mock -r opensuse-tumbleweed-x86_64 caddy-2.8.4-1.suse.tw.src.rpm

Do not forget to mention full commandline with the mock command you executed.

Any additional notes

Output of `mock --debug-config`

INFO: mock.py version 5.9 starting (python version = 3.13.0, NVR = mock-5.9-1.fc41), args: /usr/libexec/mock/mock --debug-config Start(bootstrap): init plugins INFO: selinux enabled Finish(bootstrap): init plugins Start: init plugins INFO: selinux enabled Finish: init plugins INFO: Signal handler active Start: run config_opts['additional_packages'] = [] config_opts['bootstrap_image'] = 'registry.fedoraproject.org/fedora:{{ releasever }}' config_opts['build_log_fmt_str'] = '%(message)s' config_opts['cache_alterations'] = False config_opts['chroot_name'] = 'default' config_opts['chroot_setup_cmd'] = 'install @{% if mirrored %}buildsys-{% endif %}build' config_opts['chrootuid'] = 1000 config_opts['cleanup_on_failure'] = False config_opts['cleanup_on_success'] = False config_opts['config_file'] = '/etc/mock/default.cfg' config_opts['config_path'] = '/etc/mock' config_opts['config_paths'] = ['/etc/mock/site-defaults.cfg', '/etc/mock/chroot-aliases.cfg', '/etc/mock/templates/fedora-branched.tpl', '/etc/mock/fedora-40-x86_64.cfg'] config_opts['description'] = 'Fedora {{ releasever }}' config_opts['dist'] = 'fc{{ releasever }}' config_opts['dnf.conf'] = ('\n' '[main]\n' 'keepcache=1\n' 'system_cachedir=/var/cache/dnf\n' 'debuglevel=2\n' 'reposdir=/dev/null\n' 'logfile=/var/log/yum.log\n' 'retries=20\n' 'obsoletes=1\n' 'gpgcheck=0\n' 'assumeyes=1\n' 'syslog_ident=mock\n' 'syslog_device=\n' 'install_weak_deps=0\n' 'metadata_expire=0\n' 'best=1\n' 'module_platform_id=platform:f{{ releasever }}\n' 'protected_packages=\n' 'user_agent={{ user_agent }}\n' '\n' '# repos\n' '\n' '[local]\n' 'name=local\n' 'baseurl=https://kojipkgs.fedoraproject.org/repos/f{{ releasever ' '}}-build/latest/$basearch/\n' 'cost=2000\n' 'enabled={{ not mirrored }}\n' 'skip_if_unavailable=False\n' '\n' '{% if mirrored %}\n' '[fedora]\n' 'name=fedora\n' 'metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=$basearch\n' 'gpgkey=file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-{{ ' 'releasever }}-primary\n' 'gpgcheck=1\n' 'skip_if_unavailable=False\n' '\n' '[updates]\n' 'name=updates\n' 'metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-released-f$releasever&arch=$basearch\n' 'gpgkey=file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-{{ ' 'releasever }}-primary\n' 'gpgcheck=1\n' 'skip_if_unavailable=False\n' '\n' '[updates-testing]\n' 'name=updates-testing\n' 'metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-testing-f$releasever&arch=$basearch\n' 'enabled=0\n' 'gpgkey=file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-{{ ' 'releasever }}-primary\n' 'gpgcheck=1\n' 'skip_if_unavailable=False\n' '\n' '[fedora-debuginfo]\n' 'name=fedora-debuginfo\n' 'metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-debug-$releasever&arch=$basearch\n' 'enabled=0\n' 'gpgkey=file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-{{ ' 'releasever }}-primary\n' 'gpgcheck=1\n' 'skip_if_unavailable=False\n' '\n' '[updates-debuginfo]\n' 'name=updates-debuginfo\n' 'metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-released-debug-f$releasever&arch=$basearch\n' 'enabled=0\n' 'gpgkey=file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-{{ ' 'releasever }}-primary\n' 'gpgcheck=1\n' 'skip_if_unavailable=False\n' '\n' '[updates-testing-debuginfo]\n' 'name=updates-testing-debuginfo\n' 'metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-testing-debug-f$releasever&arch=$basearch\n' 'enabled=0\n' 'gpgkey=file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-{{ ' 'releasever }}-primary\n' 'gpgcheck=1\n' 'skip_if_unavailable=False\n' '\n' '[fedora-source]\n' 'name=fedora-source\n' 'metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-source-$releasever&arch=$basearch\n' 'gpgkey=file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-{{ ' 'releasever }}-primary\n' 'gpgcheck=1\n' 'enabled=0\n' 'skip_if_unavailable=False\n' '\n' '[updates-source]\n' 'name=updates-source\n' 'metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-released-source-f$releasever&arch=$basearch\n' 'gpgkey=file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-{{ ' 'releasever }}-primary\n' 'gpgcheck=1\n' 'enabled=0\n' 'skip_if_unavailable=False\n' '{% endif %}\n') config_opts['enable_disable_repos'] = [] config_opts['extra_chroot_dirs'] = ['/run/lock'] config_opts['files'] = {'etc/hosts': '127.0.0.1 localhost localhost.localdomain\n' '::1 localhost localhost.localdomain localhost6 ' 'localhost6.localdomain6\n'} config_opts['legal_host_arches'] = ('x86_64',) config_opts['macros'] = {'%_buildhost': 'dulis16p-lan', '%_platform_multiplier': 1, '%_rpmfilename': '%%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm', '%_topdir': '/builddir/build'} config_opts['mirrored'] = True config_opts['no-config'] = {'epel-8': {'alternatives': {'alma+epel-8': {'description': ['Builds against ' 'AlmaLinux 8 ' 'repositories, ' 'together with ' 'the official ' 'EPEL ' 'repositories.', 'Project page: ' 'https://almalinux.org/']}, 'centos-stream+epel-8': {'description': ['Builds ' 'against ' 'CentOS ' 'Stream ' '8 ' 'repositories ' '(some ' 'packages ' 'may be ' 'a bit ' 'ahead ' 'the Red ' 'Hat ' 'Enterprise ' 'Linux ' '8) ' 'together ' 'with ' 'the ' 'official ' 'EPEL 8 ' 'repositories.', 'Project ' 'page: ' 'https://www.centos.org/centos-stream/']}, 'circlelinux+epel-8': {'description': ['Builds ' 'against ' 'Circle ' 'Linux 8 ' 'repositories, ' 'together ' 'with the ' 'official ' 'EPEL ' 'repositories.', 'Project ' 'page: ' 'https://cclinux.org/']}, 'rhel+epel-8': {'description': ['Builds against ' 'Red Hat ' 'Enterprise Linux ' '8 repositories, ' 'together with ' 'the official ' 'EPEL ' 'repositories.', 'This mimics what ' 'is done in the ' 'official EPEL ' 'build system, ' 'but you need a ' 'Red Hat ' 'subscription:', 'https://rpm-software-management.github.io/mock/Feature-rhelchroots']}, 'rocky+epel-8': {'description': ['Builds against ' 'Rocky Linux 8 ' 'repositories, ' 'together with ' 'the official ' 'EPEL ' 'repositories.', 'Project page: ' 'https://rockylinux.org/']}}}, 'epel-9': {'alternatives': {'alma+epel-9': {'description': ['Builds against ' 'AlmaLinux 9 ' 'repositories, ' 'together with ' 'the official ' 'EPEL ' 'repositories.', 'Project page: ' 'https://almalinux.org/']}, 'centos-stream+epel-9': {'description': ['Builds ' 'against ' 'CentOS ' 'Stream ' '9 ' 'repositories ' '(some ' 'packages ' 'may be ' 'a bit ' 'ahead ' 'the Red ' 'Hat ' 'Enterprise ' 'Linux ' '9) ' 'together ' 'with ' 'the ' 'official ' 'EPEL ' 'repositories.', 'Project ' 'page: ' 'https://www.centos.org/centos-stream/']}, 'rhel+epel-9': {'description': ['Builds against ' 'Red Hat ' 'Enterprise Linux ' '9 repositories, ' 'together with ' 'the official ' 'EPEL ' 'repositories.', 'This mimics what ' 'is done in the ' 'official EPEL ' 'build system, ' 'but you need a ' 'Red Hat ' 'subscription:', 'https://rpm-software-management.github.io/mock/Feature-rhelchroots']}, 'rocky+epel-9': {'description': ['Builds against ' 'Rocky Linux 9 ' 'repositories, ' 'together with ' 'the official ' 'EPEL ' 'repositories.', 'Project page: ' 'https://rockylinux.org/']}}}} config_opts['nspawn_args'] = ['--capability=cap_ipc_lock', '--bind=/tmp/mock-resolv.4h7hq_kl:/etc/resolv.conf', '--bind=/dev/btrfs-control', '--bind=/dev/mapper/control', '--bind=/dev/fuse', '--bind=/dev/loop-control', '--bind=/dev/loop0', '--bind=/dev/loop1', '--bind=/dev/loop2', '--bind=/dev/loop3', '--bind=/dev/loop4', '--bind=/dev/loop5', '--bind=/dev/loop6', '--bind=/dev/loop7', '--bind=/dev/loop8', '--bind=/dev/loop9', '--bind=/dev/loop10', '--bind=/dev/loop11'] config_opts['package_manager'] = '{% if releasever|int >= 40 %}dnf5{% else %}dnf{% endif %}' config_opts['plugin_conf'] = {'__init___enable': False, '__init___opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'bind_mount_enable': True, 'bind_mount_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'create_dirs': False, 'dirs': [], 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'buildroot_lock_enable': False, 'buildroot_lock_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'ccache_enable': False, 'ccache_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'compress': None, 'debug': False, 'dir': '/var/cache/mock/fedora-40-x86_64/ccache/u1000/', 'hashdir': True, 'max_cache_size': '4G', 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64', 'show_stats': False}, 'chroot_scan_enable': False, 'chroot_scan_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'only_failed': True, 'regexes': ['^[^k]?core(\\.\\d+)?$', '\\.log$'], 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64', 'write_tar': False}, 'compress_logs_enable': False, 'compress_logs_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'command': 'gzip', 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'hw_info_enable': True, 'hw_info_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'lvm_root_enable': False, 'lvm_root_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'pool_name': 'mockbuild', 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'mount_enable': True, 'mount_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'dirs': [], 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'overlayfs_enable': False, 'overlayfs_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'package_state_enable': True, 'package_state_opts': {'available_pkgs': False, 'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'installed_pkgs': True, 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'pm_request_enable': False, 'pm_request_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'procenv_enable': False, 'procenv_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'root_cache_enable': True, 'root_cache_opts': {'age_check': True, 'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'compress_program': 'pigz', 'decompress_program': None, 'dir': '/var/cache/mock/fedora-40-x86_64/root_cache/', 'exclude_dirs': ['./proc', './sys', './dev', './tmp/ccache', './var/cache/yum', './var/cache/dnf', './var/log'], 'extension': '.gz', 'max_age_days': 15, 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'rpkg_preprocessor_enable': False, 'rpkg_preprocessor_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'cmd': '/usr/bin/preproc-rpmspec %(source_spec)s ' '--output %(target_spec)s', 'requires': ['preproc-rpmspec'], 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'rpmautospec_enable': False, 'rpmautospec_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'cmd_base': ['/usr/bin/rpmautospec', 'process-distgit'], 'requires': ['rpmautospec'], 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'selinux_enable': True, 'selinux_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'showrc_enable': False, 'showrc_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'sign_enable': False, 'sign_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'cmd': 'rpmsign', 'opts': '--addsign %(rpms)s', 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'tmpfs_enable': False, 'tmpfs_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'keep_mounted': False, 'max_fs_size': None, 'mode': '0755', 'required_ram_mb': 900, 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}, 'yum_cache_enable': True, 'yum_cache_opts': {'basedir': '/var/lib/mock/fedora-40-x86_64', 'cache_topdir': '/var/cache/mock', 'cachedir': '/var/cache/mock/fedora-40-x86_64', 'max_age_days': 30, 'max_metadata_age_days': 30, 'online': True, 'resultdir': '/var/lib/mock/fedora-40-x86_64/result', 'root': 'fedora-40-x86_64'}} config_opts['print_main_output'] = True config_opts['releasever'] = '40' config_opts['repo_arch'] = 'x86_64' config_opts['root'] = 'fedora-{{ releasever }}-{{ target_arch }}' config_opts['root_log_fmt_str'] = '%(levelname)s %(filename)s:%(lineno)d: %(message)s' config_opts['rpmbuild_arch'] = 'x86_64' config_opts['state_log_fmt_str'] = '%(asctime)s - %(message)s' config_opts['target_arch'] = 'x86_64' config_opts['verbose'] = 1 Finish: run

[xsuchy]

This should be rather reported to opensuse release engineers.

[kilesduli]

This should be rather reported to opensuse release engineers.

I have identified the issue, and it is related to the mock bootstrap configuration. I pulled the opensuse/tumbleweed-dnf image and ran dnf install patterns-devel-base-devel_rpm_build, which resulted in installing systemd-presets-branding-openSUSE version 12.2-23.4. This made me realize that there is a discrepancy in the bootstrap environment, so I checked the dnf transaction history:

Transaction ID : 1
Begin time     : Sun Nov 17 19:22:51 2024
Begin rpmdb    : e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
End time       : Sun Nov 17 19:22:54 2024 (3 seconds)
End rpmdb      : b0d9db2364ee2063a9ecab5087e4be01586578da2da304f6afaa4c82e4a6a471 **
User           : System <unset>
Return-Code    : Success
Releasever     : 0
Command Line   : --config /usr/src/packages/KIWI-docker/build/image-root/kiwi_dnf4.conf -y --enableplugin=priorities --disableplugin=versionlock --installroot /usr/src/packages/KIWI-docker/build/image-root --releasever=0 --setopt=install_weak_deps=False install aaa_base bash bzip2 ca-certificates ca-certificates-mozilla coreutils cracklib-dict-small curl dnf filesystem glibc-locale-base gzip netcfg openSUSE-build-key openSUSE-release-ftp rpm-repos-openSUSE-Tumbleweed tar xz zstd

In the openSUSE bootstrap, rpm-repos-openSUSE-Tumbleweed is included, which provides the correct systemd-presets-branding-openSUSE package. However, the mock configuration does not include it. I will open a pull request to resolve this issue.