Develop for v0.23.0 #263

cflerin · 2020-12-03T11:22:15Z

Major changes:

Implement the scATAC-seq preprocessing workflow (#245):

Barcode correction [singlecelltoolkit, [NEW TOOL] [scATAC-seq] single_cell_toolkit #249]
"Debarcode" fastq files:
- 10x [single_cell_toolkit, [NEW TOOL] [scATAC-seq] single_cell_toolkit #249]
- BioRad [BAP, [NEW TOOL] [scATAC-seq] BAP #246]
Adapter Trimming [Trim Galore / Cutadapt, [NEW TOOL] [scATAC-seq] Trim Galore #242]
Mapping [bwa mem, [NEW TOOL] [scATAC-seq] BWA #243]
- mark duplicates [samtools markdup, [NEW TOOL] [scATAC-seq] BWA #243]
Generate fragments file [Sinto, [NEW TOOL] [scATAC-seq] Sinto #244]

Other scATAC-seq notes:

Currently compatible with 10x-format fastq files (R1, R2[barcode reads], R3), and BioRad (R1, R2), which are processed with BAP.
Added documentation for preprocessing steps
The scATAC-seq workflows are in main_atac.nf and need to be referenced with the full path: vib-singlecell-nf/vsn-pipelines/main_atac.nf when running the pipeline.
Added two additional input data types to getDataChannel: bam and fragments, both of which are loaded with their associated index file.

Other scRNA-seq tool changes:

celeda
- Add ability to remove empty cells after applying DecontX
- Fix plotting bug when no outliers are found
- Add cell_annotate config entry
scanpy
- Add clustering preflight checks ([SUGGESTION] Scanpy find markers step can fail when only one cluster found from clustering algorithm scanpy#48)
- Fixes to limit cpu usage in various processes.
- Adaptive threshold cell filtering threshold (Add strategy filter based on adaptive thresholds (e.g.: MAD) scanpy#47)
- Add qc plots for n_counts filter in report notebook
- Add single_sample_qc workflow
- Bug fixes for mnncorrect, clustering, pca inputs
pcacv
- Adapt *-n-pc and k parameters if violating requirement of data dimensionality
- Handle cases where no optimal number of PCs are found (Add a fallback on the number of PCs returned if no optimum found pcacv#15)
- Fix arguments passed to run_pca_cv.R
- Fix Fails when toNPCs param is higher than the feature dimension pcacv#14
sratoolkit
- Fix missing assignment operator in config

- #245

- Including handling for BioRad/ddSEQ data - #245

- #247

- relink submodules

- Add additional configs for bam, fragments - File input channel is modified to also take in the index files

- #248

- Include mapping stats

- debarcoding step is replaced with sctk - format of the metadata file changes to have all fastq files on a single line - add mark duplicates step during bwa mapping

- #251

- #245

- Updated bam/fragments input channels - Rename to qc_filtering to qc_prefilter - Update config - #248

…elWithIndex when calling it multiple times (e.g.: main_atac.nf, atac_qc_filtering). This due to the scope badly definef for some variables inside the loop of extractSample

…elWithIndex when calling it multiple times (e.g.: main_atac.nf, atac_qc_filtering). This due to the scope badly definef for some variables inside the loop of extractSample [part 2]

Co-authored-by: Kris Davie <[email protected]>

Co-authored-by: Maxime De Waegeneer <[email protected]>

- Relink all submodules. - Tool params are now scoped under `param.tools.[toolname]`. - atac metadata is now in `params.data.atac_preprocess.metadata`.

- Publish bap barcode correction log - Include barcode correction method from singlecelltoolkit

- Add barcode correction details - Fix parameter naming

Add scATAC-seq preprocessing workflow

Develop for v0.23.0 Former-commit-id: b5167f5

cflerin and others added 30 commits March 5, 2020 15:05

Link cistopic module

c349fa3

Merge branch 'develop' into develop_atac

bd7a147

Add scATAC-seq preprocessing workflow

4566a08

- #245

Link submodules for scATAC-seq preprocessing workflow

45a262f

Update scATAC preprocessing workflow

757933f

- Including handling for BioRad/ddSEQ data - #245

Addd archr submodule

c29d2d4

- #247

Updated docs for scATAC-seq

f82297e

Cleanup config includes

c4227ce

Update running times for various processes

d2659c4

- relink submodules

Add bam and fragments options to getDataChannel

253bbd8

- Add additional configs for bam, fragments - File input channel is modified to also take in the index files

Initial qc and cell calling workflow

68a94ce

- #248

Link archr submodule for qc workflow

67ca893

Relink bwamaptools module

e689c74

- Include mapping stats

Relink submodules

4708411

Fix bap module link

b66dc2e

Relink bap submodule

a0a6bc3

Change preprocess workflow to use singlecelltoolkit

f041856

- debarcoding step is replaced with sctk - format of the metadata file changes to have all fastq files on a single line - add mark duplicates step during bwa mapping

Fix preprocess config and sctx submodule

cb0335e

Relink sinto module with default barcode setting

46bc4a6

Add pycisTopic submodule

9a8e5bc

- #251

Add documentation for ATAC preprocessing steps

06fbb0a

Relink submodules after organizing output publishing

1f5e3dd

Update output publishing for atac preprocessing workflow

de02101

- #245

Initial scATAC QC prefiltering workflow

710294f

- Updated bam/fragments input channels - Rename to qc_filtering to qc_prefilter - Update config - #248

Unlink some unused submodules

d890aa2

Unlink unused submodules, pt 2

49cf7db

Relink sinto module after adding use_chrom param

f945dc8

Fix bwa index staging and update documentation

595456c

Fix bug leading to out of sync issues of the input params of getChann…

4c518b7

…elWithIndex when calling it multiple times (e.g.: main_atac.nf, atac_qc_filtering). This due to the scope badly definef for some variables inside the loop of extractSample

Fix bug leading to out of sync issues of the input params of getChann…

d2ca53d

…elWithIndex when calling it multiple times (e.g.: main_atac.nf, atac_qc_filtering). This due to the scope badly definef for some variables inside the loop of extractSample [part 2]

cflerin and others added 18 commits November 25, 2020 16:04

Cleanup bam/fragments import in channels.nf

4899056

Apply suggestions from code review to docs/scatac-seq.rst

397a38b

Co-authored-by: Kris Davie <[email protected]>

Fix var case in channels/file.nf

12d7a75

Cleanup preprocess.nf

b5ae7bb

Update conf/atac/preprocess.config

dae20f2

Co-authored-by: Maxime De Waegeneer <[email protected]>

Cleanup and rescope params for atac preprocess workflow

1c9a61c

- Relink all submodules. - Tool params are now scoped under `param.tools.[toolname]`. - atac metadata is now in `params.data.atac_preprocess.metadata`.

Relink bap and sctx modules

222c464

- Publish bap barcode correction log - Include barcode correction method from singlecelltoolkit

Add barcode correction method to atac_preprocess

9e26485

Update atac docs

1f176a8

- Add barcode correction details - Fix parameter naming

Relink bwamaptools to use longer queue on summary

4912e66

Update pcacv tool

db7c432

Update celda tool

7658586

Update sratoolkit

7058a8d

Update scanpy tool

7c4991c

Merge branch 'develop' into develop_atac

301493f

Merge pull request #252 from vib-singlecell-nf/develop_atac

4d100e8

Add scATAC-seq preprocessing workflow

Update version to 0.23.0

962a1ef

Update main readme with scATAC-seq links

c54eec5

cflerin merged commit b5167f5 into master Dec 3, 2020

KrisDavie pushed a commit that referenced this pull request Dec 8, 2020

Merge pull request #263 from vib-singlecell-nf/develop

0a585c2

Develop for v0.23.0 Former-commit-id: b5167f5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Develop for v0.23.0 #263

Develop for v0.23.0 #263

cflerin commented Dec 3, 2020

Develop for v0.23.0 #263

Develop for v0.23.0 #263

Conversation

cflerin commented Dec 3, 2020

Major changes:

Other scRNA-seq tool changes: