Running the scoring scripts:

0. Install Python requirements: `pip install -r requirements.txt`. You may
   want to run everything in a virtual environment:

    python -m virtualenv venv
    . ./venv/bin/activate
    pip install -r requirements.txt

   You'll need to activate the virtual environment every time before you run
   scoresd.py.

   If you have trouble installing MySQL-python, you can install pymysql
   instead.

1. Make sure MySQL is up and running, with a db 'scoring' with access
   to the user 'scoring' with no password (and make sure MySQL is
   listening only on localhost or has TCP connections disabled).

    create database scoring;
    create user scoring;
    grant all privileges on scoring.* to scoring;

2. `mysql -uscoring scoring < database.sql` will (re)create the tables,
   discarding any existing data.

3. Ensure that the the target directory for html exists. By default, this is a
   subdirectory `scoring` of the main repository, changed via the
   `scoring-local` config option.

4. `python scoresd.py` will start a daemon to update the db continuously
   from the logfile and milestones. This process logs to a file named
   `scoresd.log`, by default in the same directory as the daemon.

5. To stop the daemon, you can run `python scoresd.py --stop`, which will
   ensure that all currently updating pages finish. (This may take a bit of
   time.)

Other scripts:

For a one-off scoring update, you can run `python scbootstrap.py`. This is
mostly identical to running the first update pass from `scoresd.py`.

Script options
--------------

Neither scoresd.py and scbootstrap.py will do a complete player page rebuild
unless they have to. To trigger this, pass either script `-p`. (A player page
rebuild will also be triggered if the `players` directory is empty.) This may
be necessary if scoresd.py does not shut down gracefully for some reason; it
usually takes 3-4 hours for the full scoring database though. This may be
necessary if the daemon isn't shut down properly.

To run scoresd.py without daemonizing, pass it `-n`. This will not log to a
file, so is recommended only for testing. When run in this mode, the daemon
will respond to ctrl-c; the db state should be rolled back to the last commit
if this happens, and page generation may or may not succeed.

For more command line options, see `python scoresd.py --help`.

Configuration
-------------

Configuration is done via `sources.yml`. Main global options:

* `scoring-local`: the location for generated html files. If relative, this will
  be relative to the location where `scoresd.py` is run from.
  Default: `scoring`
* `scoring-base-url`: the base URL for where `scoring-local` is exposed on a
  server.
  Default: a `file://` URL reconstructed from `scoring-local`. This is useful
  for testing, but you will need to set this appropriately for running a
  server. This path should include the full path to wherever `scoring-local`
  exists on the server (this is a change from previous versions).
* `local-base`: the location for log, lock, and stop files.
  Default: `.` (i.e. the directory that scoresd.py is run from.)
* `rawdata-base`: a location for wherefiles.
  Default: `None` (this disables the wherefile functionality)
* `use-milestones`: whether to read in milestone files. For the full scoring
  database this is no longer feasible, but it may be reasonable to do for
  smaller installations. This must be set.
* `daemon-name`: used for naming log, lock, and stop files. E.g.
  `scoresd.log`.
  Default: `scoresd`. (Change from previous versions: this was was effectively
  `scoring`; but wasn't configurable without editing python.)
* `game-restrictions`: see below
  Default: `None`
* `sources`: a list of sources. This must be set.

Setting `sources`: This is a list of source descriptions. Each consists of the
following properties:

* `name`: the internal name of the source. Can be anything.
* `canonical_name`: the "official" name of the source; used both for aggregating
  multiple source entries, and indicating server names to the viewer. If unset,
  will just use `name`.
* `server_url`: the URL that the server is located at. Can be `None`.
* `base`: for remote servers, a URL base to look for logfiles at.
* `local`: for local servers, a path to look for logfiles at.
* `dormant`: set to True for remote servers that are no longer accessible;
  scoring will still use the cached logfiles if any, but no longer try to
  download.
* `logfiles`: a list of logfile paths using either `local` or `base` as a
  starting point.
* `milestones`: if the server is using milestones, a list of milestone paths.
* `morgues`: a list of morgue base paths. These allow rather complex filtering;
  see the official CAO sources.yml for examples.

Game restrictions and bans
--------------------------

In the sources yml, you can provide two kinds of bans, as properties under
`game-restrictions`:

* `botnames`: accounts associated with bots. Games from these accounts are
  tracked and stored in the database, but not shown in the realtime
  leaderboards.
* `buggy`: a list of game_keys that are buggy and should not be shown outside
  of player pages.

scoresd also has a facility for banning players. To use this, create a file
called banned_players.yml. This file should have a property `banned`, which
consists of a list of player names. This primarily affects reading now data:
if a player name is on this list, logfile entries with that player name will be
ignored entirely.

If you need to ban a player after logfile lines involving that player have been
read, add them to the list, and run `python ./scoresd.py --run-bans`. Be aware
that this executes DELETE queries on the database, so it is not undoable
without a full database rebuild. In addition, if the player appears in
tables that are limited to a certain number of slots, the system will not fill
in a new game into the blank spot created by the DELETE operations, and the
empty slot may be filled in by subsequent logfiles even if it should in
principle be filled be previous logfile lines. (This situation can also only
be corrected by a full database rebuild.)