Finish adding docstrings and checking logic

[jpmckinney]

• archive: should_archive
• s3

Check logic against:

• Data retention policy for crawled data open-contracting/deploy#153 (comment) from "To apply the criteria, for each local metadata file:"
  • Do we want to have number of files logic, or just number of bytes? Comment mentions chile_compra_releases 2018.

Check logic (in lib and tests) against Google doc:

• Working out what to do for a collection
• Test Scenarios for “Working out what to do for a collection”
• Dealing with data already on the archive server
• Removing old backups to get down to one per year later
• How to identify the correct log for a crawl?

[jpmckinney]

Need to review and/or move and/or document write_meta_data_file and write_data_file.

[jpmckinney]

Should consider splitting the run method into smaller methods.