sage --package metrics: New tool to assist discussions of the Sage distribution

[mkoeppe]

Discussions of the complexity of the Sage distribution pop up unexpectedly, as seen in

#36982 (comment), #36982 (comment), #36982 (comment), #36982 (comment), #36982 (comment), #36982 (comment), #36982 (comment)

#36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment), #36726 (comment)

#36777 (comment), #36777 (comment), #36777 (comment), #36777 (comment), #36777 (comment), #36777 (comment)

To help put such discussions on a solid factual basis, we introduce the command sage --package metrics.

$ ./sage -package metrics :standard:
has_file_distros_arch_txt=131
has_file_distros_conda_txt=216
has_file_distros_debian_txt=125
has_file_distros_fedora_txt=138
has_file_distros_gentoo_txt=181
has_file_distros_homebrew_txt=61
has_file_distros_macports_txt=129
has_file_distros_nix_txt=51
has_file_distros_opensuse_txt=146
has_file_distros_slackware_txt=25
has_file_distros_void_txt=184
has_file_patches=35
has_file_spkg_check=59
has_file_spkg_configure_m4=222
has_file_spkg_install=198
has_tarball_upstream_url=231
line_count_file_patches=22561
line_count_file_spkg_check=402
line_count_file_spkg_configure_m4=2792
line_count_file_spkg_install=2960
packages=272
type_standard=272

Use PATH=build/bin:$PATH SAGE_ROOT=some-other-worktree build/bin/sage-package metrics :standard: to obtain the metrics of another version of Sage in some other worktree.

We add computation and before/after comparison of the metrics to the CI Linux Incremental workflow.
As an illustration, we change one Python package from "normal" to "wheel", removing an spkg-install.in file in the process. See https://github.com/sagemath/sage/actions/runs/7342841283/job/19992606617?pr=36977#step:6:12

More metrics can be added after

• Refactor bootstrap-conda, m4/sage_spkg_collect.m4 through sage-package; handle dependencies_build #36740

📝 Checklist

• The title is concise, informative, and self-explanatory.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

[kwankyu]

You may want to document --package metrics in https://doc.sagemath.org/html/en/developer/packaging.html. In particular, please explain the meanings of exemplary output lines. Some of them are not obvious unless you read the code.

[mkoeppe]

Thanks for the suggestion. Done in 6e0d890

[kwankyu]

Otherwise, lgtm.

[kwankyu]

The PR description seems screwed up.

[github-actions]

Documentation preview for this PR (built with commit 6731a1c; changes) is ready! 🎉

[mkoeppe]

Thanks!