Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline remodelling. Issues 24, 31 and 36 #44

Merged
merged 173 commits into from
Mar 28, 2022
Merged

Pipeline remodelling. Issues 24, 31 and 36 #44

merged 173 commits into from
Mar 28, 2022

Conversation

fmalmeida
Copy link
Owner

@fmalmeida fmalmeida commented Dec 17, 2021

The implementation is not ready yet. This PR has been opened just to better keep track of changes made.

@fmalmeida fmalmeida changed the title Pipeline remodelling: draft PR Pipeline remodelling. Issues 24, 31 and 36 Mar 24, 2022
@fmalmeida fmalmeida merged commit 62c5417 into develop Mar 28, 2022
@fmalmeida fmalmeida deleted the remodeling branch March 28, 2022 21:53
fmalmeida added a commit that referenced this pull request Mar 30, 2022
* Pipeline remodelling. Issues 24, 31 and 36 (#44)

* Split out the config profiles (#43)

* Split out the config profiles

* Resolve the conflict using the upstream config

* trying to have a create dbs process

* found way to download dbs

* adding resfinder and plasmidfinder download rules

* added phigaro, vfdb and amrfinder download rules

* added last database download rules

* added prokka HMM download rule

* added label to use docker for tools

* fixed argminer download

* fixed victors download

* fixed iceberg download

* prokka using given database and bioconda image

* docker specific for downloading databases

* update packages

* added mlst database download rule

* add bacannot db info

* also downloads PGAP db

* added prokka and mlst

* fix conditional

* fixed pgap conditional

* added barrnap

* added 'compute_gc' module -- think about modules with two tools

* adding identation

* fixing label

* first tries on conda envs

* removing label

* adding plasmidfinder

* added platon

* working until islandpath

* added: VFDB

* added: Victors

* changing name and organization of MISC image

* added: PHAST

* added: phigaro

* added: phispy

* added: iceberg

* added: kofamscan db download

* removing named outputs as it will be solved in another PR

* added: kofamscan from downloaded database

* added: kegg_decoder

* Refactor channel identifiers and process names (#45)

**Main change**

Processes, channels and workflow names have had its case names changed in order to meet nextflow's community standards.

**Commits**
* Refactor names
* refactor names in the workflows
* Update workflows/bacannot.nf
* Update workflows/bacannot.nf
* Apply suggestions from code review
* Update workflows/bacannot.nf wrt iceberg
* Accomodate code review and fix the channel name
* fix MERGE_ANNOTATIONS process case in line 331

Co-authored-by: Felipe Marques de Almeida <[email protected]>

* trying to brind amrfinder, card_rgi and argminer

* trying to fix how the scale is calculated for amrfinder

* changing scale to perl

* typo fix

* removing unnecessary comma

* Update amrfinder.nf

passing calculation of thresold into two decimals scale to groovy

* properly working until amrfinder and card_rgi

* added small dataset test profile

* fixing name of quicktest profile

* fixing urls of testing samplesheets

* removing unnecessary files

* fix missing label

* adding resfinder to miscellaneous image

* fixing gitignore

* change manifest to upcoming version

* updating db download workflow behaviour

* removing .git dirs

* fixing argminder download

* properly added resfinder

* adding script that parses nanopolish methyl call

* removing unnecessary labels

* starting to change how images are done

* added perl tools

Added:

* prokka
* barrnap
* mlst

* added misc module

added:

* gc_compute module

* adding labels

* added kofam analysis

* added main pyenv

added tools:

* platon
* plasmidfinder

* fixed perlenv image

Added tool:

* islandpath

* included virulence modules

they use the miscellaneous image

* pyenv image updated

added prophage tools:

* phast scan
* phigaro
*phispy

* added iceberg db module

it uses the miscellaneous image

* added py36env image

added tool:

* rgi

* added resistance tools

added:

* argminer db module
* resfinder
* amrfinderplus

* added nanopolish

* added refseq_masher

* added digIS

Docker image and modules were updated to run until digIS

* added antismash

* added sequence server

* added merge_annotation module

* added draw_gis modules

* added gff2gbk module

* added create_sql module

* added first resource management labels

* fixing resouce label for phigaro

* little finx in env

* fixing how tuples should be passed

* arrived at jbrowse step

* fixing draw_gis module input tuple

* jbrowse added

* Create test_pr.yml

This is how new versions will be organised after the remodeling branch update, and, therefore, the test_pr must be updated.

* fixed phast db incorporation

* fixing inputs on report

* Update test_pr.yml

Will only act in ready for review PRs

* adding ENV VAR for current version

* adding scripts to automatically build images

* fixed iceberg db incorporation

* Update digIS.nf

DIGis gff usage fixed

* fixed vfdb incorporation

* changing file to path input resolution

* fixed victors db incorporation

* changed channel names in main script

* fixed argminer and prokka tables

* fixed custom db annotations incorporation

* fixed bacannot server loading

* fixed custom db reports

* begin incorporation of nf-core framework

* nf-core libs have been added to the pipeline

* custom database annotation added to JBrowse

* fixed custom database gff generation

* Update Dockerfile

diminished image size by removing antismash db from inside it

* Update docker.config

uncommenting assemblers images

* creating singularity profile

* begin documentation update

* added singularity profile

* made image compatible with earlier versions

* given 777 permissions to workdir

* adjusted default values

* fixed prokka to work with singularity

* fixed rgi for singularity usage

* updating PR test action

* update targeted branches

* limiting process resources

* updated image for singularity

* removing unused dbs in quicktest

* not using big dbs in quicktest

* Update resfinder.nf

* adding gitpod config

* fixed yml

* fixing custom db report getattributefileld snippet

* begin change to mkdocs

* add requirements

* Update .readthedocs.yml

* Update .readthedocs.yml

* Update .readthedocs.yml

* Update .readthedocs.yml

* update

* update requirements

* added index

* fixed tags

* added installation information

* now on quickstart

* changed some admonitions

* added quickstart

* added samplesheet page to mkdocs

* included dir with images

* Update samplesheet.md

fixed header levels

* outputs page added to mkdocs

* Update standard.config

removing "paralallel_jobs" definition

* Update defaults.config

fixed blast default values

* updated and tested quickstart

* updating gitpod.yml

* creates a testing dir with more space

* added profile selection information

* Update manual.md

manual updated for mkdocs and with new --ncbi_proteins parameter

* Update nextflow_schema.json

updated --custom_db parameter description

* Update nextflow.config

removed parameters that are in default and should not be in the boilerplate

* added config file page

* information about custom databases added

* Update nextflow_schema.json

* defaults need to be loaded before boilerplate

* changing label of unicycler and flye

* fixed antismash installation

* fixed keggdecoder requires py36

* fixed resfinder module

fixed db setup

* pipeline fixed for docker

Co-authored-by: Abhinav Sharma <[email protected]>

* removing travis CI file

* Update CHANGELOG.md

* wget tries 10 times before hanging

* fixed indentation

* database download modules retry 2 times before raising errors

* Delete update_database_image.sh

script not useful anymore

* Update manual.md

* Update README.md

* Update manual.md

* update bacannot_db information

* Update manual.md

* to avoid errors, load either pgap or tigrfam in prokka

Instead of loading both, as PGAP contains TIGRFAM, either load one or another. User selects. Default is loading TIGRFAM which is smaller.

Also updated information about it.

* Update manual.md

* updated prokka db information

* Update manual.md

* Update README.md

* Update manual.md

* Update manual.md

* Update manual.md

Co-authored-by: Abhinav Sharma <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants