# About Redist
This is a system implementing facilities to produce Quicklisp distributions. Specifically, it implements the following:

1. Scanning of ASD files to discover system definitions and dependencies, //without// having to load the ASD or any systems in question.
2. A protocol to describe the objects related to dists, projects, and releases.
3. A mechanism to serialise and restore these objects to retain the information across images.
4. Integrations with source control systems to both clone and update project source files.
5. Parsers for the "quicklisp-projects"(https://github.com/quicklisp/quicklisp-projects) information, making it possible to create a dist just like the official "quicklisp" dist.
6. A compiler that takes the project source files and dist information to produce the tarballs and metadata files Quicklisp expects for a dist. Ultimately you can just point an HTTP server at the output directory and your dist setup is complete.
7. A testing system to allow compiling each of the systems in a dist release, and verify that they still perform as expected.

## Quickstart (Binary)
You can download a precompiled binary of Redist from the ''releases''(https://github.com/shirakumo/dist/releases/latest), or create it yourself by cloning Redist and using:

::
sbcl --eval '(asdf:make :redist)'
::

You should end up with a ``redist`` binary. When you invoke it, it should spill out a help that explains all the options and parameters. For a quickstart, first create your dist:

::
./redist add-dist my-dist --url "http://my.public.url"
::

Then create the projects you want to add:

::
./redist add https://github.com/someone/my-project.git
./redist add https://someone.com/project.tgz
::

This will create a ``sources/`` directory where each project's source files are kept.

Finally you can compile the dist files:

::
./redist compile -v
::

Once your dist has been compiled, it'll have created a ``releases/`` directory. You can simply copy this file to a static HTML server or serve them directly using nginx or similar.

In the future to create a new dist release, you can simply invoke:

::
./redist compile -vu
::

To update all the projects, first. The ``-v`` flag prints output. You can also add ``-j 4`` or similar to speed up the process by running it on multiple threads.

## Systemd Service
On most Linux systems you can set up a Systemd service file like ``/etc/systemd/system/redist.service``

::
[Unit]
Description=Redist Distribution Compilation
After=network.target

[Service]
Type=oneshot
ExecStart=/home/redist/redist compile -vuj 4
WorkingDirectory=/home/redist
User=redist
Group=redist
::

Assuming there's a ``redist`` user that holds the ``redist`` binary and all the data. You can then trigger a recompile via ``systemctl start redist``.

If you want to automatically rather than manually trigger compiles, you can also create a timer file like ``/etc/systemd/system/redist.timer``

::
[Unit]
Description=Redist Releases

[Timer]
OnCalendar=monthly
Persistent=true

[Install]
WantedBy=timers.target
::

And enable it via ``systemctl enable redist.timer``. Now an automatic release will be triggered every month.

## Quickstart (Lisp)
Creating your own dist is rather simple:

:: common lisp
(redist:define-project my-project
    ((:git "https://github.com/someone/my-project.git")))

(redist:define-project other-project
    ((:http "https://someone.com/project.tgz")))

(redist:define-dist my-dist (my-project other-project)
  :url "http://my.public.url")
::

This will create the ``dist`` object including ``project``s and clone their sources to disk. If there's any problems with the cloning process, it will signal an error and allow you to continue in several ways.

Once a dist is set up, you'll want to serialise its current configuration to disk to ensure you can restore it at a later point when you want to update it.

:: common lisp
(redist:store T T T)
(redist:retrieve T T T)
::

This will save all the information to disk in a ``distinfo.lisp`` file and an accompanying directory of bits. The directory structure is important to allow it to load information lazily as it's needed. Over time, the entire history of a dist is stored that way, which can grow quite large. Lazy loading is important to make sure startup remains snappy.

Once you have the dist set up with all the projects you'd like, it is time to make a release. To do so, simply call ``compile``.

:: common lisp
(redist:compile 'my-dist)
::

This will produce the necessary tarballs and metadata files. All you have to do now to make it accessible, is to point an HTTP server at the URL you declared earlier to the ``releases`` directory. Note that as of now, the Quicklisp client cannot talk HTTPS, so you must make your server accessible over plain HTTP.

Once a dist release has been produced, you should ``persist`` the changes, to ensure that when you create a new release it can share archives of projects that have not updated, and that it can properly regenerate the dist version index.

Should you want to add or remove projects from a dist later, you can simply redefine your dist, or use the functions ``add-project`` and ``remove-project``, if you prefer. You can also maintain multiple dists with different sets of projects and different release schedules. The dists will share the archives whenever project versions coincide.

In order to update the project sources, you can either individually ``update`` a project, or pass ``:update T`` to ``compile``. Additionally, if you'd prefer some extra output, since creating a release can take a while, you can also pass ``:verbose T``.

If you would like to produce a dist from the official "quicklisp-projects"(https://github.com/quicklisp/quicklisp-projects), simply clone that repository and load its data in via ``parse-quicklisp-projects``. Be careful though, cloning all the repositories takes quite a while, and a few of the links have since broken.

## HTML
Along with the actual dist files, Redist also generates a set of HTML. Specifically, it does so for:

- The set of dists / projects you serve
- Each dist
- Each project
- Each release
- Each project release

This should help people explore the stuff you host on your servers without having to actually install the dist. It also helps to have a more interactive way to search through versions.

If you would like to customise how the pages look, please have a look at the files in the ``template/`` directory. The HTML pages are generated using "Clip"(https://shinmera.github.io/clip), and the CSS is generated using "LASS"(https://shinmera.github.io/LASS).

## Programs
The following programs should be accessible for some features of Redist to operate:

- ``curl``
  For sources hosted as bare archives and replicating dists
- ``bsdtar``
  For sources hosted as bare archives and replicating dists
- ``git``
  For sources hosted in Git repositories
- ``hg``
  For sources hosted in Mercurial repositories
- ``darcs``
  For sources hosted in Darcs repositories
- ``svn``
  For sources hosted in Subversion repositories
- ``cvs``
  For sources hosted in CVS repositories