Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 28 additions & 15 deletions src/doc/en/developer/coding_basics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,9 @@ In particular,
Files and directory structure
=============================

Roughly, the Sage directory tree is layout like this. Note that we use
``SAGE_ROOT`` in the following as a shortcut for the (arbitrary) name
of the directory containing the Sage sources:
Roughly, the Sage directory tree is laid out like this. Note that we
use ``SAGE_ROOT`` in the following as a shortcut for the name of the
directory containing the Sage sources:

.. CODE-BLOCK:: text

Expand Down Expand Up @@ -153,8 +153,8 @@ Adding new top-level packages below :mod:`sage` should be done
sparingly. It is often better to create subpackages of existing
packages.

Non-Python Sage source code and supporting files can be included in one
of the following places:
Non-Python Sage source code and small supporting files can be
included in one of the following places:

- In the directory of the Python code that uses that file. When the
Sage library is installed, the file will be installed in the same
Expand All @@ -165,16 +165,6 @@ of the following places:

os.path.join(os.path.dirname(__file__), 'sage-maxima.lisp')

- Large data files should not be added to the Sage source tree.
Instead:

- Create a separate git repository for them
- Add metadata in your repository that make it a pip-installable
package (distribution package)
- Upload it to PyPI
- Create metadata in ``SAGE_ROOT/build/pkgs`` that make your new
pip-installable package known to Sage

- In an appropriate subdirectory of ``SAGE_ROOT/src/sage/ext_data/``.
(At runtime, it is then available in the directory indicated by
``SAGE_EXTCODE``). For example, if ``file`` is placed in
Expand All @@ -188,6 +178,29 @@ the section ``options.package_data`` of the file
``SAGE_ROOT/pkgs/sagemath-standard/setup.cfg.m4`` (or the corresponding
file of another distribution).

Large data files should not be added to the Sage source tree. Instead, it
is proposed to do the following:

- create a separate git repository and upload them there [2]_,

- add metadata to the repository that make it a pip-installable
package (distribution package),

- upload it to PyPI,

- create metadata in ``SAGE_ROOT/build/pkgs`` that make your new
pip-installable package known to Sage.

For guiding examples of external repositories that host large data
files, see https://github.com/sagemath/conway-polynomials, and
https://github.com/gmou3/matroid-database.

.. [2]

It is also suggested that the files are compressed, e.g., through
the command ``xz -e``. They can then be read via a command such as
``lzma.open(file, 'rt')``.


Learn by copy/paste
===================
Expand Down