Installation fails if specifying file system DarkPAN mirror after official one

[maxhq]

The problem is that a mirror might return an HTML error page which is saved as .tar.gz. It cannot be unpacked, so fetch_module tries the next mirror. If this is a local filesystem DarkPAN and the file exists, it is not copied again in file_mirror because the target already exists (although with invalid data). Unpack fails again.

Possible solution:

1. Delete invald archives if unpacking fails
2. Always overwrite target files just as the HTTP based mirroring methods do (HTTP::Tinyish::*)

Reproduce:

• specify mirrors and
• try to install a module that is only available in a local file sytem mirror and
• specify an official CPAN mirror before the local one.

Example:

/update/vendor/darkpan/authors/id/D/DU/DUMMY/Beetle-0.30.tar.gz exists and is valid.

This works, but obviously only if no other modules (specified in the cpanfile) are installed before.

cpanm --mirror /update/vendor/darkpan --mirror http://cpan.metacpan.org --verbose --notest -L /update/local --installdeps ./

while this fails:

cpanm --mirror http://cpan.metacpan.org --mirror /update/vendor/darkpan --verbose --notest -L /update/local --installdeps ./

Console output

cpanm (App::cpanminus) 1.7044 on perl 5.012005 built for x86_64-linux
Work directory is /root/.cpanm/work/1544654866.8214
You have make /usr/bin/make
You have /usr/bin/wget
You have /bin/tar: tar (GNU tar) 1.26
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by John Gilmore and Jay Fenlason.
You have /usr/bin/unzip
--> Working on ./
Entering /update
Configuring /update ... OK
Checking if you have DateTime::Format::HTTP == 0.38 ... Yes (0.38)
Checking if you have Crypt::Blowfish == 2.10 ... Yes (2.10)
Checking if you have Unicode::String == 2.09 ... Yes (2.09)
Checking if you have MooseX::Singleton == 0.30 ... Yes (0.30)
...
Checking if you have Beetle == 0.30 ... No
==> Found dependencies: Beetle
Searching Beetle (== 0.30) on cpanmetadb ...
--2018-12-12 23:47:46--  http://cpanmetadb.plackperl.org/v1.0/history/Beetle
Resolving cpanmetadb.plackperl.org (cpanmetadb.plackperl.org)... 151.101.66.217, 151.101.130.217, 151.101.194.217, ...
Connecting to cpanmetadb.plackperl.org (cpanmetadb.plackperl.org)|151.101.66.217|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2018-12-12 23:47:46 ERROR 404: Not Found.

--2018-12-12 23:47:46--  http://fastapi.metacpan.org/v1/download_url/Beetle?version=%3D%3D%200.30
Resolving fastapi.metacpan.org (fastapi.metacpan.org)... 151.101.66.217, 151.101.130.217, 151.101.194.217, ...
Connecting to fastapi.metacpan.org (fastapi.metacpan.org)|151.101.66.217|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2018-12-12 23:47:46 ERROR 404: Not Found.

! Could not find a release matching Beetle (== 0.30) on MetaCPAN.
Searching Beetle on mirror http://cpan.metacpan.org ...
Downloading index file http://cpan.metacpan.org/modules/02packages.details.txt.gz ...
--2018-12-12 23:47:46--  http://cpan.metacpan.org/modules/02packages.details.txt.gz
Resolving cpan.metacpan.org (cpan.metacpan.org)... 151.101.130.217, 151.101.194.217, 151.101.2.217, ...
Connecting to cpan.metacpan.org (cpan.metacpan.org)|151.101.130.217|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2138645 (2.0M) [application/x-gzip]
Saving to: `/root/.cpanm/sources/http%cpan.metacpan.org/02packages.details.txt.gz'

100%[====================================================================================================================================================>] 2,138,645   2.86M/s   in 0.7s    

2018-12-12 23:47:48 (2.86 MB/s) - `/root/.cpanm/sources/http%cpan.metacpan.org/02packages.details.txt.gz' saved [2138645/2138645]

! Finding Beetle (== 0.30) on mirror http://cpan.metacpan.org failed.
Searching Beetle on mirror file:///update/vendor/darkpan ...
Downloading index file file:///update/vendor/darkpan/modules/02packages.details.txt.gz ...
--> Working on Beetle
Fetching http://cpan.metacpan.org/authors/id/D/DU/DUMMY/Beetle-0.30.tar.gz ... --2018-12-12 23:47:48--  http://cpan.metacpan.org/authors/id/D/DU/DUMMY/Beetle-0.30.tar.gz
Resolving cpan.metacpan.org (cpan.metacpan.org)... 151.101.194.217, 151.101.2.217, 151.101.66.217, ...
Connecting to cpan.metacpan.org (cpan.metacpan.org)|151.101.194.217|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2018-12-12 23:47:48 ERROR 404: Not Found.

OK
Unpacking Beetle-0.30.tar.gz

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Error is not recoverable: exiting now
! Failed to unpack Beetle-0.30.tar.gz: no directory
Fetching file:///update/vendor/darkpan/authors/id/D/DU/DUMMY/Beetle-0.30.tar.gz ... OK
Unpacking Beetle-0.30.tar.gz

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Error is not recoverable: exiting now
! Failed to unpack Beetle-0.30.tar.gz: no directory
! Failed to fetch distribution Beetle-0.30
! Installing the dependencies failed: Module 'Beetle' is not installed
! Bailing out the installation for ./.