Make snakemake a dev dependency

[huddlej]

Technically augur does not depend on snakemake to run and including it here can
cause confusion and/or issues installing either augur or snakemake. This commit
removes the snakemake dependency and leaves the installation of snakemake up to
the user.

[codecov]

Codecov Report

Merging #557 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #557   +/-   ##
=======================================
  Coverage   28.31%   28.31%           
=======================================
  Files          39       39           
  Lines        5332     5332           
  Branches     1306     1306           
=======================================
  Hits         1510     1510           
  Misses       3769     3769           
  Partials       53       53           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8b6f950...412ac7b. Read the comment docs.

[tsibley]

100% agree we should remove this hard dep, but I think first we want to

• Update https://github.com/nextstrain/conda to include Snakemake with the same version spec, so that the environment remains "batteries included"
• Audit and address Augur's and nextstrain.org's docs for places which assume installing Augur means you have Snakemake.

[huddlej]

That sounds great to me! I can post a PR for point one tomorrow evening.

The second point is going to take more time, but maybe @eharkins has a better sense of which docs would need to be changed since his recent deep dive into all Nextstrain docs.

[eharkins]

Happy to include

Audit and address Augur's and nextstrain.org's docs for places which assume installing Augur means you have Snakemake.

in my general look at docs and try to come up with a comprehensive list of places where that assumption is made. The tutorials sort of assume this although the zika tutorial does explain that snakemake is necessary and how it is used with augur.

There is also some duplication of explaining how snakemake works with augur, e.g.:

• in the zika tutorial
• in faq
• in usage

Deduplicating that will involve some design decisions about how we want the tutorials to use snakemake (see this thread on slack).

[prakashvishnoi]

snakemake dependency removed as desired.

[eharkins]

Here are all of the places I could find in docs where either snakemake is referred to (U) or the use of snakemake is assumed as I interpret it (A):

Augur docs:

• (U) https://docs.nextstrain.org/projects/augur/en/stable/releases/migrating-v5-v6.html?highlight=snakemake
• (U) https://docs.nextstrain.org/projects/augur/en/stable/index.html?highlight=snakemake
• (U) https://docs.nextstrain.org/projects/augur/en/stable/examples/examples.html?highlight=snakemake
• (U) https://docs.nextstrain.org/projects/augur/en/stable/usage/augur_snakemake.html?highlight=snakemake (duplicated in:

1. (U) https://docs.nextstrain.org/projects/augur/en/stable/faq/augur_snakemake.html?highlight=snakemake and
2. (A) https://nextstrain-augur.readthedocs.io/en/stable/tutorials/zika_tutorial.html#chaining-of-augur-commands-with-snakemake )

• (A) https://docs.nextstrain.org/projects/augur/en/stable/tutorials/tb_tutorial.html?highlight=snakemake
• (A) https://docs.nextstrain.org/projects/augur/en/stable/faq/clades.html?highlight=snakefile
• (A) https://docs.nextstrain.org/projects/augur/en/stable/usage/cli/parse.html?highlight=rule
• (A) https://docs.nextstrain.org/projects/augur/en/stable/usage/cli/refine.html?highlight=rule
• (A) https://docs.nextstrain.org/projects/augur/en/stable/usage/cli/filter.html?highlight=rule
• (A) https://github.com/nextstrain/augur/blob/a6291c4a44e20d31cfc8535780b4f4cdfb689354/environment.yml
• (A) https://github.com/nextstrain/augur/tree/master/tests/builds
• (A) https://github.com/nextstrain/augur/blob/2ff315fe319e7713b417ce6b4bf51bb42e4c9f35/DEV_DOCS.md

nextstrain.org docs:

• (U) here is a good thing to link to (and perhaps update according to removed dependency) regarding use of snakemake https://nextstrain.org/docs/bioinformatics/what-is-a-build#snakemake
• (U) zika + tb tutorials (https://github.com/nextstrain/nextstrain.org/tree/master/static-site/content/docs/03-tutorials - hosted here as opposed to the ones on augur RTD) do a good job of prefacing the snakemake sections with:

While it is instructive to run all of the above commands manually, it is more practical to automate their execution with a single script. Nextstrain implements these automated pathogen builds with Snakemake by defining a Snakefile like the one in the Zika repository you downloaded.

• (A) https://nextstrain.org/docs/tutorials/defining-clades

nextstrain cli
Does nextstrain cli get snakemake from augur as a dependency (it seems to be needed for nextstrain cli)? If so, does it need to be added as an explicit dependency of nextstrain cli or is that already addressed by:

Update https://github.com/nextstrain/conda to include Snakemake with the same version spec, so that the environment remains "batteries included"

?

[tsibley]

Re: the CLI and Snakemake: I don't think there will be any updates needed to the CLI if Snakemake is removed from Augur's deps, because the Docker image already includes it and we'd update the Conda environment to do so too.

[rneher]

most of our tests use Snakemake. we would have to add it as a dev dependency.

[huddlej]

Update https://github.com/nextstrain/conda to include Snakemake with the same version spec, so that the environment remains "batteries included"

This is addressed in conda PR 8.

Thank you, @eharkins, for collecting all the relevant docs! I'll look through these and update any mentions of augur/Snakemake that need it.

[huddlej]

It's getting hard to keep track of all the moving pieces required for this PR to go in, so I've tried to make a comprehensive checklist below.

• Update docs to describe required Snakemake --cores argument
  • augur (Document snakemake commands with required cores argument #599)
  • nextstrain.org (Document required cores argument for snakemake and nextstrain build nextstrain.org#207)
• Update docs to describe that Snakemake must be installed separately
  • augur (Document augur's external dependency on Snakemake #600)
  • nextstrain.org (no updates needed!)
• Update Nextstrain’s conda environment to include Snakemake (Add Snakemake as a dependency for the Nextstrain environment conda#8)
• Update Snakemake pinned version in augur dev dependencies
• Remove Snakemake as an augur dependency (Make snakemake a dev dependency #557)

After reviewing the nextstrain.org docs, I don't think we need to make any additional changes. These docs expect the user to install augur/auspice through conda or Docker. For both of these approaches, Snakemake will already be installed even if it isn't installed as part of augur's installation. The augur PR 600 updates the pip-based instructions to mention that Snakemake must be installed separately.