improve portability of reproducible tarballs by replacing external tar command with tarfile module from Python standard library

[lexming]

fixes #4657

• use more portable --date argument for touch
• catch failed commands inside the pipeline
• move generation of command to make reproducible archives intro its own method
• replace harcoded pattern in tests of reproducible archives command for call to filetools.reproducible_archive_cmd
• add required argument to filetools.find_extensions()
• make new implementation of reproducible_archive_cmd using the tarfile module
• added new filetools.make_archive() method and related unit test
• change handling of filename argument in filetools.get_source_tarball_from_git() to allow any extension and pass it to make_archive()
• make_archive() now supports uncompressed tarballs and compressed tarballs in GZIP, BZIP2 and XZ
• make_archive() can create reproducible tarballs in .tar or .tar.gz format
• checksums of sources from git repos will be verified in Python 3.9+

[lexming]

@boegel This one is ready. As discussed, archives will be made with tarfile instead of the tar command and checksums of sources from git repos will only be checked with Python 3.9+.

[boegel]

@lexming We need to make sure that git_config entries that still use .tar.gz as extension don't leading to clone+tar+xz every time, we also need to check whether the source filename with .tar.xz already exists?

[boegel]

TODO here:

• change so function that creates tarballs for git_config sources honors the provided file extension;
• only .tar.xz can be produced reproducibly (and only using Python >= 3.9);

[lexming]

@boegel this is ready on my side, summary of the final behaviour:

• (no change) git_config continues to provide filename with extension
• (enhancement) get_source_tarball_from_git will respect file extension
• (enhancement) checksums of sources in git_config will be verified whenever:
  1. checksum is provided in easyconfig
  2. Python 3.9+
• (enhancement) regular archives of git repos can now be created for uncompressed .tar or compressed tarballs in GZIP, XZ and BZIP2
• (enhancement) reproducible archives are supported for:
  1. Uncompressed tarball or compressed with XZ
  2. Python 3.9+

This means that existing easyconfigs with .tar.gz sources and no checksums will continue to behave in the same way (no verification) across all Python versions. Only when a checksum is manually added, it will trigger checksum verification on Python 3.9+.

We can migrate existing easyconfigs to .tar.xz and include a checksum to start using reproducible tarballs. Users on older Pythons will see a deprecation warning and fallback to current behaviour (no verification) and installation will proceed.

[boegel]

lgtm