Update the pipeline to address reviewers comments [ci skip] (#10)

* Reinitialize pipeline with nf-core template [ci skip] * install relevant modules [ci skip] * add the fully concatenated file [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * download assemblies using ncbi-datasets-cli [ci skip] * add more references for reviewer-assemblies [ci skip] * add download script [ci skip] * dev iteration [ci skip] * use baseline java8 for orthoani [ci skip] * add orthoani util modules [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * update the hardware req for trimmomatic [ci skip] * add ggdc results [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * restore spades in favor of util process for filtering [ci skip] * add post-spades utils [ci skip] * add post-spades utils [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * make fastgrep executable [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * test with latest checkm process [ci skip] * retry with 1.1.8 checkm and reduced tree [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * dev iteration [ci skip] * regenrate the ggdc results [ci skip] * add params and orthoani subworkflow [ci skip] * use a custom container [ci skip] * use lower resources [ci skip] * add raxml [ci skip] * add raxml [ci skip] * add raxml [ci skip] * add raxml [ci skip] * add raxml [ci skip] * add orthofinder [ci skip] * add orthofinder [ci skip] * add orthofinder [ci skip] * add orthofinder [ci skip] * iterate on orthofinder [ci skip] * iterate on orthofinder [ci skip] * iterate on orthofinder [ci skip] * iterate on orthofinder [ci skip] * iterate on orthoani [ci skip]
emilyncosta · Aug 7, 2023 · 88827f9 · 88827f9
1 parent b4fc0fd
commit 88827f9
Show file tree

Hide file tree

Showing 121 changed files with 48,896 additions and 1,869 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1,2 +1,4 @@
 results*
 data
+null
+run_analysis.sh
diff --git a/.nf-core.yml b/.nf-core.yml
@@ -0,0 +1 @@
+repository_type: pipeline
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -0,0 +1,16 @@
+# UNIFESP_LABMICOBACT/ntm_mterrae_nf: Changelog
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## v1.0dev - [date]
+
+Initial release of UNIFESP_LABMICOBACT/ntm_mterrae_nf, created with the [nf-core](https://nf-co.re/) template.
+
+### `Added`
+
+### `Fixed`
+
+### `Dependencies`
+
+### `Deprecated`
diff --git a/CITATIONS.md b/CITATIONS.md
@@ -0,0 +1,35 @@
+# UNIFESP_LABMICOBACT/ntm_mterrae_nf: Citations
+
+## [nf-core](https://pubmed.ncbi.nlm.nih.gov/32055031/)
+
+> Ewels PA, Peltzer A, Fillinger S, Patel H, Alneberg J, Wilm A, Garcia MU, Di Tommaso P, Nahnsen S. The nf-core framework for community-curated bioinformatics pipelines. Nat Biotechnol. 2020 Mar;38(3):276-278. doi: 10.1038/s41587-020-0439-x. PubMed PMID: 32055031.
+
+## [Nextflow](https://pubmed.ncbi.nlm.nih.gov/28398311/)
+
+> Di Tommaso P, Chatzou M, Floden EW, Barja PP, Palumbo E, Notredame C. Nextflow enables reproducible computational workflows. Nat Biotechnol. 2017 Apr 11;35(4):316-319. doi: 10.1038/nbt.3820. PubMed PMID: 28398311.
+
+## Pipeline tools
+
+- [FastQC](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/)
+
+- [MultiQC](https://pubmed.ncbi.nlm.nih.gov/27312411/)
+  > Ewels P, Magnusson M, Lundin S, Käller M. MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics. 2016 Oct 1;32(19):3047-8. doi: 10.1093/bioinformatics/btw354. Epub 2016 Jun 16. PubMed PMID: 27312411; PubMed Central PMCID: PMC5039924.
+
+## Software packaging/containerisation tools
+
+- [Anaconda](https://anaconda.com)
+
+  > Anaconda Software Distribution. Computer software. Vers. 2-2.4.0. Anaconda, Nov. 2016. Web.
+
+- [Bioconda](https://pubmed.ncbi.nlm.nih.gov/29967506/)
+
+  > Grüning B, Dale R, Sjödin A, Chapman BA, Rowe J, Tomkins-Tinch CH, Valieris R, Köster J; Bioconda Team. Bioconda: sustainable and comprehensive software distribution for the life sciences. Nat Methods. 2018 Jul;15(7):475-476. doi: 10.1038/s41592-018-0046-7. PubMed PMID: 29967506.
+
+- [BioContainers](https://pubmed.ncbi.nlm.nih.gov/28379341/)
+
+  > da Veiga Leprevost F, Grüning B, Aflitos SA, Röst HL, Uszkoreit J, Barsnes H, Vaudel M, Moreno P, Gatto L, Weber J, Bai M, Jimenez RC, Sachsenberg T, Pfeuffer J, Alvarez RV, Griss J, Nesvizhskii AI, Perez-Riverol Y. BioContainers: an open-source and community-driven framework for software standardization. Bioinformatics. 2017 Aug 15;33(16):2580-2582. doi: 10.1093/bioinformatics/btx192. PubMed PMID: 28379341; PubMed Central PMCID: PMC5870671.
+
+- [Docker](https://dl.acm.org/doi/10.5555/2600239.2600241)
+
+- [Singularity](https://pubmed.ncbi.nlm.nih.gov/28494014/)
+  > Kurtzer GM, Sochat V, Bauer MW. Singularity: Scientific containers for mobility of compute. PLoS One. 2017 May 11;12(5):e0177459. doi: 10.1371/journal.pone.0177459. eCollection 2017. PubMed PMID: 28494014; PubMed Central PMCID: PMC5426675.
diff --git a/LICENSE b/LICENSE
@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2020 Bioinformatics Playground
+Copyright (c) abhi18av
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

diff --git a/Makefile b/Makefile
diff --git a/README.md b/README.md
@@ -1,39 +1,92 @@
-# camila_sao_paulo nextflow pipeline
-A pipeline for Genome Assembly, Genome Anotation and Variant Calling with quality evaluation, using `.fastq` files and a reference genome as input.
+[![Cite with Zenodo](http://img.shields.io/badge/DOI-10.5281/zenodo.XXXXXXX-1073c8?labelColor=000000)](https://doi.org/10.5281/zenodo.XXXXXXX)
 
-## Minimal requirements (for local execution)
+[![Nextflow](https://img.shields.io/badge/nextflow%20DSL2-%E2%89%A522.10.1-23aa62.svg)](https://www.nextflow.io/)
+[![run with conda](http://img.shields.io/badge/run%20with-conda-3EB049?labelColor=000000&logo=anaconda)](https://docs.conda.io/en/latest/)
+[![run with docker](https://img.shields.io/badge/run%20with-docker-0db7ed?labelColor=000000&logo=docker)](https://www.docker.com/)
+[![run with singularity](https://img.shields.io/badge/run%20with-singularity-1d355c.svg?labelColor=000000)](https://sylabs.io/docs/)
+[![Launch on Nextflow Tower](https://img.shields.io/badge/Launch%20%F0%9F%9A%80-Nextflow%20Tower-%234256e7)](https://tower.nf/launch?pipeline=https://github.com/UNIFESP_LABMICOBACT/ntm_mterrae_nf)
 
-* Nextflow 
-* Java
-* Docker
+## Introduction
 
-## Quick start
+**UNIFESP_LABMICOBACT/ntm_mterrae_nf** is a bioinformatics pipeline that ...
 
-### Local execution
-1. Install nextflow 
+<!-- TODO nf-core:
+   Complete this sentence with a 2-3 sentence summary of what types of data the pipeline ingests, a brief overview of the
+   major pipeline sections and the types of output it produces. You're giving an overview to someone new
+   to nf-core here, in 15-20 seconds. For an example, see https://github.com/nf-core/rnaseq/blob/master/README.md#introduction
+-->
 
-	Please refer to [Nextflow page on github](https://github.com/nextflow-io/nextflow/) for more info.
+<!-- TODO nf-core: Include a figure that guides the user through the major workflow steps. Many nf-core
+     workflows use the "tube map" design for that. See https://nf-co.re/docs/contributing/design_guidelines#examples for examples.   -->
+<!-- TODO nf-core: Fill in short bullet-pointed list of the default steps in the pipeline -->
 
-2. Clone it 
+1. Read QC ([`FastQC`](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/))
+2. Present QC for raw reads ([`MultiQC`](http://multiqc.info/))
 
+## Usage
 
-```shell
-git clone https://github.com/emilyncosta/labmicobact_unifesp_ntm_romagnoli_nf
-```
+> **Note**
+> If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/usage/installation) on how
+> to set-up Nextflow. Make sure to [test your setup](https://nf-co.re/docs/usage/introduction#how-to-run-a-pipeline)
+> with `-profile test` before running the workflow on actual data.
+
+<!-- TODO nf-core: Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.
+     Explain what rows and columns represent. For instance (please edit as appropriate):
 
-3. Run it!
+First, prepare a samplesheet with your input data that looks as follows:
 
-Please update the `params/local.yml` file before using it in local execution.
+`samplesheet.csv`:
 
+```csv
+sample,fastq_1,fastq_2
+CONTROL_REP1,AEG588A1_S1_L002_R1_001.fastq.gz,AEG588A1_S1_L002_R2_001.fastq.gz
 ```
-nextflow run main.nf -params-file params/local.yml 
+
+Each row represents a fastq file (single-end) or a pair of fastq files (paired end).
+
+-->
+
+Now, you can run the pipeline using:
+
+<!-- TODO nf-core: update the following command to include all required parameters for a minimal example -->
+
+```bash
+nextflow run UNIFESP_LABMICOBACT/ntm_mterrae_nf \
+   -profile <docker/singularity/.../institute> \
+   --input samplesheet.csv \
+   --outdir <OUTDIR>
 ```
 
+> **Warning:**
+> Please provide pipeline parameters via the CLI or Nextflow `-params-file` option. Custom config files including those
+> provided by the `-c` Nextflow option can be used to provide any configuration _**except for parameters**_;
+> see [docs](https://nf-co.re/usage/configuration#custom-configuration-files).
+
+## Credits
+
+UNIFESP_LABMICOBACT/ntm_mterrae_nf was originally written by abhi18av.
+
+We thank the following people for their extensive assistance in the development of this pipeline:
+
+<!-- TODO nf-core: If applicable, make list of people who have also contributed -->
+
+## Contributions and Support
+
+If you would like to contribute to this pipeline, please see the [contributing guidelines](.github/CONTRIBUTING.md).
+
+## Citations
+
+<!-- TODO nf-core: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file. -->
+<!-- If you use  UNIFESP_LABMICOBACT/ntm_mterrae_nf for your analysis, please cite it using the following doi: [10.5281/zenodo.XXXXXX](https://doi.org/10.5281/zenodo.XXXXXX) -->
 
-## Configuration Profiles.
+<!-- TODO nf-core: Add bibliography of tools and data used in your pipeline -->
 
-You can use diferent profiles for this pipeline, based on the computation enviroment at your disposal. Here are the Avaliable Profiles in `conf/` folder.
+An extensive list of references for the tools used by the pipeline can be found in the [`CITATIONS.md`](CITATIONS.md) file.
 
-## License
+This pipeline uses code and infrastructure developed and maintained by the [nf-core](https://nf-co.re) community, reused here under the [MIT license](https://github.com/nf-core/tools/blob/master/LICENSE).
 
-All Nextflow code within the repo is licensed under the liberal MIT license. For the code used in `bin/` and `resources/`, we defer the to same license as used by the original authors.
+> **The nf-core framework for community-curated bioinformatics pipelines.**
+>
+> Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.
+>
+> _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x).
diff --git a/assets/adaptivecard.json b/assets/adaptivecard.json
@@ -0,0 +1,67 @@
+{
+    "type": "message",
+    "attachments": [
+        {
+            "contentType": "application/vnd.microsoft.card.adaptive",
+            "contentUrl": null,
+            "content": {
+                "\$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
+                "msteams": {
+                    "width": "Full"
+                },
+                "type": "AdaptiveCard",
+                "version": "1.2",
+                "body": [
+                    {
+                        "type": "TextBlock",
+                        "size": "Large",
+                        "weight": "Bolder",
+                        "color": "<% if (success) { %>Good<% } else { %>Attention<%} %>",
+                        "text": "UNIFESP_LABMICOBACT/ntm_mterrae_nf v${version} - ${runName}",
+                        "wrap": true
+                    },
+                    {
+                        "type": "TextBlock",
+                        "spacing": "None",
+                        "text": "Completed at ${dateComplete} (duration: ${duration})",
+                        "isSubtle": true,
+                        "wrap": true
+                    },
+                    {
+                        "type": "TextBlock",
+                        "text": "<% if (success) { %>Pipeline completed successfully!<% } else { %>Pipeline completed with errors. The full error message was: ${errorReport}.<% } %>",
+                        "wrap": true
+                    },
+                    {
+                        "type": "TextBlock",
+                        "text": "The command used to launch the workflow was as follows:",
+                        "wrap": true
+                    },
+                    {
+                        "type": "TextBlock",
+                        "text": "${commandLine}",
+                        "isSubtle": true,
+                        "wrap": true
+                    }
+                ],
+                "actions": [
+                    {
+                        "type": "Action.ShowCard",
+                        "title": "Pipeline Configuration",
+                        "card": {
+                            "type": "AdaptiveCard",
+                            "\$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
+                            "body": [
+                                {
+                                    "type": "FactSet",
+                                    "facts": [<% out << summary.collect{ k,v -> "{\"title\": \"$k\", \"value\" : \"$v\"}"}.join(",\n") %>
+                                    ]
+                                }
+                            ]
+                        }
+                    }
+                ]
+            }
+        }
+    ]
+}
diff --git a/assets/email_template.html b/assets/email_template.html
@@ -0,0 +1,53 @@
+<html>
+<head>
+  <meta charset="utf-8">
+  <meta http-equiv="X-UA-Compatible" content="IE=edge">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+
+  <meta name="description" content="UNIFESP_LABMICOBACT/ntm_mterrae_nf: A Nextflow pipeline for M. terrae analysis">
+  <title>UNIFESP_LABMICOBACT/ntm_mterrae_nf Pipeline Report</title>
+</head>
+<body>
+<div style="font-family: Helvetica, Arial, sans-serif; padding: 30px; max-width: 800px; margin: 0 auto;">
+
+<img src="cid:nfcorepipelinelogo">
+
+<h1>UNIFESP_LABMICOBACT/ntm_mterrae_nf v${version}</h1>
+<h2>Run Name: $runName</h2>
+
+<% if (!success){
+    out << """
+    <div style="color: #a94442; background-color: #f2dede; border-color: #ebccd1; padding: 15px; margin-bottom: 20px; border: 1px solid transparent; border-radius: 4px;">
+        <h4 style="margin-top:0; color: inherit;">UNIFESP_LABMICOBACT/ntm_mterrae_nf execution completed unsuccessfully!</h4>
+        <p>The exit status of the task that caused the workflow execution to fail was: <code>$exitStatus</code>.</p>
+        <p>The full error message was:</p>
+        <pre style="white-space: pre-wrap; overflow: visible; margin-bottom: 0;">${errorReport}</pre>
+    </div>
+    """
+} else {
+    out << """
+    <div style="color: #3c763d; background-color: #dff0d8; border-color: #d6e9c6; padding: 15px; margin-bottom: 20px; border: 1px solid transparent; border-radius: 4px;">
+        UNIFESP_LABMICOBACT/ntm_mterrae_nf execution completed successfully!
+    </div>
+    """
+}
+%>
+
+<p>The workflow was completed at <strong>$dateComplete</strong> (duration: <strong>$duration</strong>)</p>
+<p>The command used to launch the workflow was as follows:</p>
+<pre style="white-space: pre-wrap; overflow: visible; background-color: #ededed; padding: 15px; border-radius: 4px; margin-bottom:30px;">$commandLine</pre>
+
+<h3>Pipeline Configuration:</h3>
+<table style="width:100%; max-width:100%; border-spacing: 0; border-collapse: collapse; border:0; margin-bottom: 30px;">
+    <tbody style="border-bottom: 1px solid #ddd;">
+        <% out << summary.collect{ k,v -> "<tr><th style='text-align:left; padding: 8px 0; line-height: 1.42857143; vertical-align: top; border-top: 1px solid #ddd;'>$k</th><td style='text-align:left; padding: 8px; line-height: 1.42857143; vertical-align: top; border-top: 1px solid #ddd;'><pre style='white-space: pre-wrap; overflow: visible;'>$v</pre></td></tr>" }.join("\n") %>
+    </tbody>
+</table>
+
+<p>UNIFESP_LABMICOBACT/ntm_mterrae_nf</p>
+<p><a href="https://github.com/UNIFESP_LABMICOBACT/ntm_mterrae_nf">https://github.com/UNIFESP_LABMICOBACT/ntm_mterrae_nf</a></p>
+
+</div>
+
+</body>
+</html>
diff --git a/assets/email_template.txt b/assets/email_template.txt
@@ -0,0 +1,31 @@
+Run Name: $runName
+
+<% if (success){
+    out << "## UNIFESP_LABMICOBACT/ntm_mterrae_nf execution completed successfully! ##"
+} else {
+    out << """####################################################
+## UNIFESP_LABMICOBACT/ntm_mterrae_nf execution completed unsuccessfully! ##
+####################################################
+The exit status of the task that caused the workflow execution to fail was: $exitStatus.
+The full error message was:
+
+${errorReport}
+"""
+} %>
+
+
+The workflow was completed at $dateComplete (duration: $duration)
+
+The command used to launch the workflow was as follows:
+
+  $commandLine
+
+
+
+Pipeline Configuration:
+-----------------------
+<% out << summary.collect{ k,v -> " - $k: $v" }.join("\n") %>
+
+--
+UNIFESP_LABMICOBACT/ntm_mterrae_nf
+https://github.com/UNIFESP_LABMICOBACT/ntm_mterrae_nf
diff --git a/assets/methods_description_template.yml b/assets/methods_description_template.yml
@@ -0,0 +1,25 @@
+id: "UNIFESP_LABMICOBACT-ntm_mterrae_nf-methods-description"
+description: "Suggested text and references to use when describing pipeline usage within the methods section of a publication."
+section_name: "UNIFESP_LABMICOBACT/ntm_mterrae_nf Methods Description"
+section_href: "https://github.com/UNIFESP_LABMICOBACT/ntm_mterrae_nf"
+plot_type: "html"
+## TODO nf-core: Update the HTML below to your prefered methods description, e.g. add publication citation for this pipeline
+## You inject any metadata in the Nextflow '${workflow}' object
+data: |
+  <h4>Methods</h4>
+  <p>Data was processed using UNIFESP_LABMICOBACT/ntm_mterrae_nf v${workflow.manifest.version} ${doi_text} of the nf-core collection of workflows (<a href="https://doi.org/10.1038/s41587-020-0439-x">Ewels <em>et al.</em>, 2020</a>).</p>
+  <p>The pipeline was executed with Nextflow v${workflow.nextflow.version} (<a href="https://doi.org/10.1038/nbt.3820">Di Tommaso <em>et al.</em>, 2017</a>) with the following command:</p>
+  <pre><code>${workflow.commandLine}</code></pre>
+  <h4>References</h4>
+  <ul>
+    <li>Di Tommaso, P., Chatzou, M., Floden, E. W., Barja, P. P., Palumbo, E., & Notredame, C. (2017). Nextflow enables reproducible computational workflows. Nature Biotechnology, 35(4), 316-319. <a href="https://doi.org/10.1038/nbt.3820">https://doi.org/10.1038/nbt.3820</a></li>
+    <li>Ewels, P. A., Peltzer, A., Fillinger, S., Patel, H., Alneberg, J., Wilm, A., Garcia, M. U., Di Tommaso, P., & Nahnsen, S. (2020). The nf-core framework for community-curated bioinformatics pipelines. Nature Biotechnology, 38(3), 276-278. <a href="https://doi.org/10.1038/s41587-020-0439-x">https://doi.org/10.1038/s41587-020-0439-x</a></li>
+  </ul>
+  <div class="alert alert-info">
+    <h5>Notes:</h5>
+    <ul>
+      ${nodoi_text}
+      <li>The command above does not include parameters contained in any configs or profiles that may have been used. Ensure the config file is also uploaded with your publication!</li>
+      <li>You should also cite all software used within this run. Check the "Software Versions" of this report to get version information.</li>
+    </ul>
+  </div>
diff --git a/assets/multiqc_config.yml b/assets/multiqc_config.yml
@@ -0,0 +1,12 @@
+report_comment: >
+  This report has been generated by the <a href="https://github.com/UNIFESP_LABMICOBACT/ntm_mterrae_nf" target="_blank">UNIFESP_LABMICOBACT/ntm_mterrae_nf</a>
+  analysis pipeline.
+report_section_order:
+  "UNIFESP_LABMICOBACT-ntm_mterrae_nf-methods-description":
+    order: -1000
+  software_versions:
+    order: -1001
+  "UNIFESP_LABMICOBACT-ntm_mterrae_nf-summary":
+    order: -1002
+
+export_plots: true
diff --git a/assets/samplesheet.csv b/assets/samplesheet.csv
@@ -0,0 +1,6 @@
+sample,fastq_1,fastq_2
+MYC101,MYC101_R1.fastq.gz,MYC101_R2.fastq.gz
+MYC123,MYC123_R1.fastq.gz,MYC123_R2.fastq.gz
+MYC17,MYC17_R1.fastq.gz,MYC17_R2.fastq.gz
+MYC340,MYC340_R1.fastq.gz,MYC340_R2.fastq.gz
+MYC98,MYC98_R1.fastq.gz,MYC98_R2.fastq.gz