Rework download/extract code in sage-spkg

[jdemeyer]

The current code in sage-spkg to determine which package to install and whether to download it is buggy and too complicated. Also get rid of calls to the newest_version script, get rid of the "download again" part (why was it needed?).

Changes in behaviour:

1. When specifying a download URL, it will always download the file, even if a corresponding spkg exists locally. This allows for easier development of spkgs, because people often post updated spkgs with the same version number.
2. "sage -f package" without a version number will use the most recent (by modification time) local spkg in spkg/standard or spkg/optional. Only if no such package exists, it will try to download it. This makes it consistent with "sage -f package-x.y.z" with a version number.
3. Allow gzip compression (in addition to bzip2 and no compression) of spkgs.

Apply attachment: 12602_spkg_download.patch to the SAGE_ROOT repository.

Depends on #12479

CC: @jhpalmieri

Component: build

Author: Jeroen Demeyer

Reviewer: John Palmieri

Merged: sage-5.0.beta10

Issue created by migration from https://trac.sagemath.org/ticket/12602

[jhpalmieri]

comment:1

Questions: if you type sage -i pkg, should it first check spkg/standard (and spkg/optional, spkg/experimental) to see if there is a matching spkg, before trying to download something? Or should it check version numbers to see if the on-line file is more recent? If it checks first in spkg/..., then should there be an option to force downloading, or should we just assume that people will figure out what's going on and delete the local version if they want to download the spkg?

[jdemeyer]

comment:2

I still think the default should be to give priority to local packages.

We could support the following (-u for "upgrade"):

./sage -i -u mpfr

which would check the latest online version of mpfr, download it if needed. What do you think?

[jdemeyer]

Description changed:

--- 
+++ 
@@ -1 +1 @@
-The current code in `sage-spkg` to determine which package to install and whether to download it is too complicated (and probably buggy).  Also get rid of calls to the `newest_version` script and allow gzip compression of .spkg files.
+The current code in `sage-spkg` to determine which package to install and whether to download it is too complicated (and probably buggy).  Also get rid of calls to the `newest_version` script, allow gzip compression of .spkg files, get rid of the "download again" part (why was it needed?).

[jdemeyer]

Changed dependencies from #10479, #10579 to #12479

[jdemeyer]

Author: Jeroen Demeyer

[jdemeyer]

Description changed:

--- 
+++ 
@@ -1 +1,3 @@
 The current code in `sage-spkg` to determine which package to install and whether to download it is too complicated (and probably buggy).  Also get rid of calls to the `newest_version` script, allow gzip compression of .spkg files, get rid of the "download again" part (why was it needed?).
+
+There is one important change in behaviour: when specifying a download URL, it will always download the file, even if a corresponding spkg exists locally.  I think this is more in line with expectations.

[jdemeyer]

Description changed:

--- 
+++ 
@@ -1,3 +1,5 @@
-The current code in `sage-spkg` to determine which package to install and whether to download it is too complicated (and probably buggy).  Also get rid of calls to the `newest_version` script, allow gzip compression of .spkg files, get rid of the "download again" part (why was it needed?).
+The current code in `sage-spkg` to determine which package to install and whether to download it is buggy and too complicated.  Also get rid of calls to the `newest_version` script, allow gzip compression of .spkg files, get rid of the "download again" part (why was it needed?).
 
-There is one important change in behaviour: when specifying a download URL, it will always download the file, even if a corresponding spkg exists locally.  I think this is more in line with expectations.
+Changes in behaviour:
+1. When specifying a download URL, it will always download the file, even if a corresponding spkg exists locally.  This allows for easier development of spkgs, because people often post updated spkgs with the same version number.
+2. "sage -f package" without a version number will use the most recent (by modification time) local spkg in spkg/standard or spkg/optional.  Only if no such package exists, it will try to download it.  This makes it consistent with "sage -f package-x.y.z" with a version number.

[jdemeyer]

Description changed:

--- 
+++ 
@@ -1,5 +1,6 @@
-The current code in `sage-spkg` to determine which package to install and whether to download it is buggy and too complicated.  Also get rid of calls to the `newest_version` script, allow gzip compression of .spkg files, get rid of the "download again" part (why was it needed?).
+The current code in `sage-spkg` to determine which package to install and whether to download it is buggy and too complicated.  Also get rid of calls to the `newest_version` script, get rid of the "download again" part (why was it needed?).
 
 Changes in behaviour:
 1. When specifying a download URL, it will always download the file, even if a corresponding spkg exists locally.  This allows for easier development of spkgs, because people often post updated spkgs with the same version number.
 2. "sage -f package" without a version number will use the most recent (by modification time) local spkg in spkg/standard or spkg/optional.  Only if no such package exists, it will try to download it.  This makes it consistent with "sage -f package-x.y.z" with a version number.
+3. Allow gzip compression (in addition to bzip2 and no compression) of spkgs.

[simon-king-jena]

comment:9

To what repository is your patch supposed to be added? It apparently does not work with devel/sage.