Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

umicollapse #1368

Closed
wants to merge 807 commits into from
Closed

umicollapse #1368

wants to merge 807 commits into from

Conversation

siddharthab
Copy link

  • Bump pipeline version to 3.15.0dev
  • update qualimap/rnaseq
  • bump versions
  • prettier
  • update CHANGELOG
  • Apply suggestions from code review
  • Move main workflow into it's own directory
  • Remove nfcore_external_java_deps.jar
  • Install utils subworkflows from nf-core modules
  • Delete lib directory and replace with utils_* subworkflows
  • Fix linting
  • Add prepare_tool_indices logic to rnaseq workflow
  • Move rrna-db-defaults.txt to workflow assets
  • Replace local MultiQC module with one from nf-core/modules
  • Remove test_cache profile
  • Add empty nextflow.config to all modules/subworkflows/workflows
  • Rename quantify_pseudo to quantify_pseudo_alignment
  • First pass updates to split config out of modules.config
  • Delete empty nextflow.config files
  • Template update for nf-core/tools version 2.12
  • Refactor configs to add if/else logic back in
  • Delete modules.config
  • [automated] Fix linting with Prettier
  • Remove nf-core modules path diff files
  • Update all nf-core modules and subworkflows
  • Update CHANGELOG
  • Fix channel renaming in rseqc subworkflow
  • Add config for publishing pseudo-alignment files
  • [automated] Fix linting with Prettier
  • Fix ECLint
  • Update .nf-core.yml to exclude linting for params.ribo_database_manifest
  • Fix bloody prettier again
  • Update README.md
  • Fix rseqc channel renaming
  • Fix rseqc channel renaming
  • Fix ECLint
  • Update CHANGELOG
  • Update nf-core/utils_nfcore_pipeline subworkflow
  • Use tx2gene from nf-core/modules
  • Bump tx2gene
  • update bam_rseqc to fix linting issue
  • Revert "update bam_rseqc to fix linting issue"
  • update bam_rseqc to fix linting issue
  • Correct for new bam_rseqc outputs
  • Remove unused local module files
  • update all modules
  • fix bam_rseqc usage
  • update utils_nfcore_pipeline subworkflows
  • fix usage of UTILS_NFCORE_PIPELINE
  • update CHANGELOG
  • update more modules
  • Next phase of nf-core pseudo module integration
  • Basic working
  • Fix configs to maintain consistency with current outputs
  • Also fix template config
  • Update changelog
  • Pass transcriptome fasta through to samtools stats
  • update changelog
  • Bump umitools and delocalise prepareforrsem
  • Update CHANGELOG
  • Fix module import
  • delocalise catadditionalfasta
  • Fix umitools cardinality
  • remove local module
  • Update changelog
  • appease eclint
  • Fix syntax
  • prettier
  • Install module
  • Fix channel shape
  • Remove test line
  • Fix map order
  • Ensure value channels
  • mention issues in changelog
  • chore: Emiller88 => edmundmiller
  • Template update for nf-core/tools version 2.13
  • Update multiqc module to fix linting
  • Update CHANGELOG
  • Update all nf-core modules and subworkflows
  • Refactor input provided to rnaseq workflow
  • Remove unused ch_versions from PIPELINE_INITIALISATION
  • Fix config selectors
  • Use more specific process selector for samtools processes
  • Module updates
  • Readd config
  • nf-test at the pipeline level
  • linting
  • CHANGELOG
  • Add ci-cd workflow for nf-test to existing nf-core testing framework
  • Tidy up CI workflow a bit
  • Increase minimum minor Nextflow version
  • Let tests fail before completing
  • Revert to minimum required Nextflow version
  • Remove latest version to make tests more efficient
  • add temporary exclusion for tests
  • no need for prettier on these folder
  • fix SUMMARIZEDEXPERIMENT_SUMMARIZEDEXPERIMENT
  • temporarily disable old tests
  • remove vs comment
  • update custom/catadditionalfasta subread/featurecounts
  • custom/tx2gene
  • Fix config selector for bbsplit indexing
  • Fix config selector for bbsplit indexing
  • Try to not overwrite
  • Try not to overwrite-better
  • Install module with hack
  • install from master
  • Update CHANGELOG
  • update configs
  • modules update: bbmap/bbsplit multiqc rseqc/all sortmerna untar
  • update exclusion
  • update more modules and remove exclusions
  • Template update for nf-core/tools version 2.13.1
  • fix hisat2 tests
  • fix linting
  • fix FASTQ_SUBSAMPLE_FQ_SALMON
  • restore even more excluded tests
  • Make README usage consistent with docs/usage.md
  • Update README.md
  • fix linting
  • Update .github/PULL_REQUEST_TEMPLATE.md
  • update subwokflows/modules and exclusion
  • update CHANGELOG
  • re-enable prepareforrsem
  • let's remove all exclusion
  • update modules/subworkflows
  • fix UTILS
  • fix gffread and stringtie_stringtie tests
  • syntax fix
  • keep components from nf-core out of prettierignore due to patch possibility
  • update umitools
  • only snapshot versions and assert success
  • restore old tests
  • fix umitools/dedup
  • update sortmerna functionalities
  • Update modules/nf-core/sortmerna/nextflow.config
  • empty index
  • add meta map
  • fix cardinality
  • typo:tyop
  • update CHANGELOG
  • fix path
  • Update subworkflows/local/prepare_genome/main.nf
  • copy tests from star_genomegenerate
  • more descriptive tests' names
  • add nf-tests for star_align_igenomes
  • Switch to genomecov from nf-core (with separate sort)
  • Fix config import
  • push tests
  • Add missing inputs, closures
  • Remove trailing whitespace
  • genomecov from modules
  • pretter on modules.json
  • Fix tests
  • Bump modules
  • Fix final test data path- update modules.json later
  • Update CHANGELOG
  • Final genomecov test fix
  • Add nf-tests for nf-core/rnaseq utility functions
  • Adds tests for nf-core/rnaseq utility subworkflows
  • Fix using schema variable correctly in PIPELINE_COMPLETION
  • test tidy
  • Add nf-test to local module GTF_FILTER
  • Update subworkflows/local/utils_nfcore_rnaseq_pipeline/main.nf
  • Fix concurrency error in Github CI workflow
  • Add nf-test to preprocess_transcripts_fasta_gencode
  • Switch to generating custom FASTA file with gencode annotations
  • bump snap
  • Update task name in snap
  • fix snap
  • add basic tests
  • proper tests
  • properly fix genome params usage
  • Add nf-tests for deseq2_qc local module
  • deseq2_qc: Move test data into separate files
  • Swtich to dupradar from nf-core
  • Fix config import
  • Update changelog
  • Install module from master
  • delete script from old local module
  • update CHANGELOG
  • nf-test for module MULTIQC_CUSTOM_BIOTYPE
  • update dupradar
  • Make name more informative
  • Add nf-test for local subworkflow QUANTIFY_RSEM
  • Add gtf2bed tests
  • last tweaks
  • Update tags.yml
  • Add nf-test to quantify_rsem local subworkflow
  • remove glob
  • Remove publishDir directive
  • Update snapshot
  • Don't snapshot logs
  • fix linting
  • Apply suggestions from code review
  • remove data
  • proper path
  • fix path
  • regenerate snapshot
  • Add nf-test for quantify_psueudoalignment
  • Some improvements but not working :(
  • uncomment the comment
  • Swap to nf-core rnaseq test data (full path currently)
  • Reference genome path mistake
  • remove unnecessary tags
  • prepare_genome: add basic nf-test tests
  • update samtools/sort module
  • fix samtools/sort usage
  • update local snaps
  • Fix bugs in quantify_pseudoalignment
  • fixup
  • Update to use samplesheet in test-datasets repo
  • Omit variable files from snap
  • Initial attempt at nf-test for rsem_merge_counts
  • Include test data
  • [skip ci] update readme
  • Update usage.md
  • Remove all tags.yml files
  • Replace deseq2qc paths
  • delete data files
  • Fix test data syntax for deseq2_qc module
  • use a local getGenomeAttribute function
  • CHANGELOG
  • Update modules/local/rsem_merge_counts/tests/main.nf.test
  • data to github
  • fix path
  • Add new testing strategy based on nf-test files
  • CHANGELOG
  • Update modules/local/deseq2_qc/tests/main.nf.test
  • one idea
  • Replace check if no failure with success()
  • Update CHANGELOG.md
  • SALMON tests keepDuplicates
  • collect input files
  • Update modules/local/rsem_merge_counts/tests/main.nf.test
  • forgot to commit this file deletion
  • Update modules/local/rsem_merge_counts/tests/main.nf.test
  • Update CHANGELOG
  • Add more tests for PREPARE_GENOME
  • CHANGELOG
  • Remove existing tests that are duplicated by PUBLISH_GENOME tests
  • Try to fix CI pipeline AGAIN
  • Check for empty list
  • Revert needs statement
  • Trying something else...
  • Add everything to test dep
  • Fixup: Add GHA files back into include statement of Github workflow change detection
  • Add missing software updates to CHANGELOG
  • Move aliased bedtools genomecov selectors to main workflow config
  • Syntax updates
  • Rename checkSamplesAreConsistent to checkSamplesAfterGrouping
  • Update CHANGELOG
  • Fix pre-commit lint
  • Update tests to omit snapshot of variable files
  • Revert renaming UMITOOLS_PREPAREFORSALMON to UMITOOLS_PREPAREFORRSEM
  • Delete unecessary tags from nt.test files for modules
  • Delete unecessary tags from nt.test files for subworkflows
  • Update CHANGELOG
  • Remove indexing on output channels
  • Update snap
  • Bug fixes and improvements
  • Remove R from Snap
  • Update trimming subworkflow to include more tests
  • Bump subworkflow version
  • move getGenomeAttribute to main.nf
  • Add direct replacements for existing tests
  • Reorganised into discrete folders
  • More all tests into subdirectory for more organisation
  • remove redundant tests
  • Remove invalid dependencies from GHA confirm-pass
  • Correct pipeline test path
  • Relative path AGAIN
  • Update bam_markduplicates_picard subworkflow
  • Delocalise pseudo quant workflow
  • Update changelog
  • Bump tximport for reproducible versions
  • Bump subworkflow for linting
  • Restore missing files
  • Strip tags.ymls
  • Remove r-base from software versions
  • Move repo tests config to tests/config
  • Add test config
  • Add psueudoaligner pipeline level tests to test suite
  • CI fixup
  • Correct snaps for pseudoalignment
  • Reorganise pipeline tests into flat structure
  • Use profile test --input parameter
  • indentation
  • Fixup CHANGELOG
  • Increase contents of default.main.nf.test.snap
  • CHANGELOG
  • Remove variable files from default pipeline snapshot
  • update snap
  • Remove more from snaps
  • feat(igenomes): update name as in template
  • feat(references): fix usage of igenomes references
  • no need for key
  • remove tests for functionned removed
  • Improve the extra_args parameter consolidation so it can override presets from the config.
  • Changelog update.
  • Implement Jonathan Manning's map-method for extra argument consolidation for the local STAR alignment subworkflow.
  • Implement Jonathan Manning's map-method for extra argument consolidation for the FastQC-UMItools-Trimgalore subworkflow..
  • Apply @pinin4fjords (Jonathan Manning)'s suggestions from code review
  • fix(subworkflow): update utils_nfcore_pipeline
  • Template update for nf-core/tools version 2.14.0
  • Template update for nf-core/tools version 2.14.1
  • Update CHANGELOG
  • Update tximport subworkflow and module
  • Update Changelog
  • Remove extra unnecessary tpm outputs
  • Update CHANGELOG
  • Update CHANGELOG
  • Check RSeQC strandedness without reference to undetermined
  • Update subworkflows/local/utils_nfcore_rnaseq_pipeline/main.nf
  • close Clarify infer strandedness from current subsampling + infer step #1095
  • Document FASTP sampling
  • Update CHANGELOG
  • Add missing subworkflow config import
  • correctly reference quantification subworkflow config
  • update changelog
  • Fix issues with unzipping of GTF/ GFF files
  • Update changelog
  • Same for other zipped reference inputs
  • Add reference files section to usage docs
  • Adjust line breaks
  • fix strand message
  • Strip problematic ifEmpty()
  • Update CHANGELOG
  • Apply suggestions from code review
  • Move reference guidance to existing sections
  • Update Salmon
  • Add strandedness detection threshold parameter
  • Add a consistent library type check between Salmon and RSeQC. Make the strand check results standard
  • Amend for undetermined
  • Constraint strand detection threshold
  • Consistent defaults
  • Reinstate onComplete messages
  • Appease linting
  • update CHANGELOG
  • strandedness failure doesn't skip
  • Fix PIPELINE_COMPLETEION test
  • Don't need non-custom completion summary
  • Fix up after merge
  • Fix for linting
  • update CHANGELOG
  • Fix the fix
  • fix typos
  • [automated] Fix linting with Prettier
  • Fix module lint error
  • revert config changed in error
  • Fix conditionality
  • Clarify stranded/ unstraded at library level, include other library types in strand totals
  • Allow for missing keys in salmon library counts for testing
  • Fix function tests
  • Update ci.yml
  • Auto updating snapshot broke things for unaffected tests. So manually insert snapshots from the only tests I do want to update
  • Add existence checks
  • correct order
  • Return percentages for strandedness
  • Explicitly publish lib_format_counts
  • update docs
  • Fix typo
  • Prettier
  • Fix typo
  • Use Github Action to detect file changes instead of custom Python code
  • Update CHANGELOG.md
  • Update actions/checkout to v4
  • Set maximum on bars per column
  • Pass/fail in separate mqc column
  • Fix multiqc config for cell colors to highlight strandedness
  • Fix strand status conditional
  • update strand check image
  • Fix multiqc yaml
  • fix up params + usage
  • Update CHANGELOG
  • Reorder genome reference params
  • Fix tags in modules
  • Fix tags in subworkflows
  • Rename pipeline level tests
  • Update CHANGELOG with new parameters
  • Change number param to have decimal place
  • Fix spacing
  • Fix syntax consistency
  • Fix prettier
  • Update CHANGELOG
  • Apply suggestions from code review
  • Minor tidying to usage docs
  • Run pre-commit
  • Move config block back where it was
  • Fix multiqc message
  • Fix params issues
  • Update mqc strand screenshot
  • Update CHANGELOG
  • Rename pipeline level tests
  • Move Conda dependencies for local modules to individual environment file
  • Update CHANGELOG
  • appease prettier
  • Update usage
  • Remove tags from all nf-test files
  • Add nf-test for STAR-RSEM and HISAT2 aligners
  • remove tags
  • CHANGELOG
  • test naming conventions
  • Update CHANGELOG
  • Update tests/hisat2.main.nf.test
  • Update tests/star_rsem.main.nf.test
  • Rename files in line with other tests
  • Update all nf-core modules and subworkflows
  • Fix pipeline after updates to GFFREAD module
  • Update CHANGELOG
  • Update snapshots for prepare_genome subworkflow
  • Remove problem multiqc setting
  • Move note about basic annotations to GENCODE section
  • Revert "Remove problem multiqc setting"
  • add stub for local/deseq2_qc
  • add stubs for all + test for most
  • Update all nf-core/modules again
  • Rename custom content keys as required in MultiQC v1.22.3
  • Update snapshots for default pipeline test
  • patch dupradar
  • Fix syntax
  • update snapshot
  • Update CHANGELOG.md
  • update snaps
  • update all modules
  • update all subworkflows
  • no more tags.yml
  • no more tags.yml
  • remove tag within tests
  • update CHANGELOG
  • stub for align_star + a single snapshot match per test
  • add stubs + code polish + Harshil align
  • add stub
  • update all nf-core modules and subworkflows and do not include tags
  • simplify snapshot for stub
  • add stubs for subworkflow local quantify_rsem
  • update default snapshot
  • Modify FastQC output publishing to comply with the paths in MultiQC config.
  • MultiQC config updates.
  • Linting.
  • Use the correct module names to supress the Percent Duplicate column in the General Stats table of MultiQC.
  • Publish logfiles from umi-tools dedup steps.
  • Modify subworkflow bam_dedup_stats_samtools_umitools to publish dedup logs.
  • Update bam_dedup_stats_samtools_umitools subworkflow and MultiQC module.
  • Add UMI information for Watchmaker mRNA Library Prep Kit.
  • Publish the UMI-tools extract logs for MultiQC.
  • The output channel of fastq_fastqc_umitools_trimgalore/fastp is called 'umi_log' and not 'log'. The meta.yml is wrong.
  • Mark FASTQC outputs with suffixes
  • @aanil (Anandashankar Anil) correctly pointed out that some MultiQC configuration parameters used in the config were deprecated.
  • Linting
  • Changelog
  • add plugin
  • update all subworkflows
  • do not snapshot fastq in stub
  • pipeline level stub
  • fix stub test for prepare_genome
  • update all subworkflows
  • correctly align
  • Use nf-core/setup-nf-test action for portability
  • changelog
  • update all modules
  • Undo changes to modules/nf-core/dupradar/templates/dupradar.r from Harshil Patel and me, since they should be fixed upstream in the modules directory. Reset state of file to cc1bf2c.
  • Update Azure Batch guidance
  • Add suggestions from @MatthiasZepper
  • bump module and temporarily patch for Update dupradar.r according to new MultiQC (>=v1.22) config values modules#5943
  • Correct the diff
  • Update MultiQC to v1.23
  • Demote titles to return structure to previous
  • Update dupradar from modules repo and remove patch.
  • use seqera containers
  • Update module for snapshot fix
  • Fix prefixes for FASTQC
  • update all modules/subworkflows + remove tags
  • Apply a blanket renaming in multiqc to catch inconsistent naming paired and unpaired
  • Switch renaming to exact matches
  • update snapshot
  • Bump multiqc module
  • Separate rename patterns for forward and reverse
  • update CHANGELOG
  • Remove Azure specific guidance
  • Update CHANGELOG.md
  • simplify logic to generate renaming table
  • Update CHANGELOG
  • Make name replacement specific to samples without tech reps
  • Clarify comment
  • Update main.nf
  • Update CHANGELOG.md
  • Factor out preprocessing logic to nf-core subworkflow
  • update changelog
  • Update RNAseq preprocessing swf
  • update modules.json
  • fix up ribo db wiring
  • Install swf from branch for now
  • Fix tests
  • Update modules.json
  • Move strandedness function testing to swf
  • Install from modules master
  • Strip preprocessing components relocated to subworkflows
  • Fix the new call to preprocessing subworkflow
  • Fix the new call to preprocessing subworkflow
  • Fix the new call to preprocessing subworkflow
  • Remove print
  • update changelog
  • update nf-test to 0.9.0
  • sort
  • update all modules/subworkflows + remove tags
  • remove tags
  • keep card 3.1.1 that does not break CI due to weird RG
  • update snapshot
  • update snapshot
  • update all modules + subworkflows / remove tags + path markduplicates
  • update snapshots
  • update stringtie_stringtie
  • update snapshots for fastq_align_hisat2
  • add proper stubs and tests for local star_*_igenomes modules
  • Bump resource bucket for genomecov
  • Revert "Bump resource bucket for genomecov"
  • Update bedtools/genomecov for lower-resource sort
  • Update changelog
  • Try multiqc fixes
  • Fix rename patterns
  • Import tsv func from nf-core subworkflow
  • Better define ordering
  • Add new header, define some sections, reinstate trimming status reporting
  • Fix up methods and versions passed to MultiQC
  • Attempt a rational fixed ordering of sections
  • Temp fix pending suworkflow update
  • update subworkflow from nf-core
  • update changelog
  • correct conditional for salmon indexing in proprocessing workflow
  • Update CHANGELOG.md
  • proper sha for module bedtools/genomecov
  • update samtools/stats module + rm tags
  • Update multiqc for prefix usage
  • tiny updates on subway map
  • realign
  • tiny fixes
  • update CHANGELOG
  • Update CHANGELOG.md
  • Bump multiqc
  • Unset multiqc prefix for testing
  • Update workflows/rnaseq/main.nf
  • clean up stub tests
  • Fix anchor issue in multiqc
  • Update CHANGELOG.md
  • update CHANGELOG
  • Animate subway map
  • update snapshot
  • update snapshot
  • Update test_full.config to restore a static URI for megatests
  • Update CHANGELOG.md
  • use permalink
  • Same for minimal tests
  • Update CHANGELOG.md
  • fix double reference
  • Fix input
  • Revert multiqc workaround due to fix
  • Update changelog
  • Update changelog
  • snapshot all files contents by default
  • animate metro map
  • fix MultiQC paths
  • Install properly
  • Tidy up multiqc test config
  • Add module config
  • Update changelog
  • Install changes from modules
  • update test and snapshot
  • update MultiQC paths
  • sort files
  • update test and snapshot
  • update test and snapshot
  • more snapshots
  • update test and snapshot
  • update CHANGELOG
  • update test and snapshot
  • update test and snapshot
  • revert update CHANGELOG
  • update test and snapshot
  • update CHANGELOG
  • text to path
  • typo
  • more snapshots
  • text to path
  • update snapshots
  • fastqc html are not stable
  • update tests and snapshots
  • update tests and snapshots
  • update tests and snapshots
  • update tests and snapshots
  • kallisto
  • Add link to static version
  • kallisto with snapshots
  • kallisto updated snapshots
  • min_mapped_reads snapshots
  • min_mapped_reads better snapshots
  • fix stub snap
  • remove_ribo_rna snapshots
  • update remove_ribo_rna snapshots
  • salmon snapshots + update moar snapshots
  • skip_qc snapshots
  • update tests and snapshots
  • fix tests and snapshots
  • skip_trimming test and snapshots
  • update test and snapshots
  • fix linting + star_rsem tests and snapshots
  • update CHANGELOG
  • fix merge conflicts
  • update tests
  • update tests and snapshots
  • update fastqc module
  • update utils subworkflows
  • use proper script for dupradar
  • no need for tags
  • no need for tags
  • update CHANGELOG
  • Add umicollapse as an alternative to umi-tools

pinin4fjords and others added 30 commits June 21, 2024 09:41
Update all nf-core/modules and subworkflows
Copy link

github-actions bot commented Sep 3, 2024

This PR is against the master branch ❌

  • Do not close this PR
  • Click Edit and change the base to dev
  • This CI test will remain failed until you push a new commit

Hi @siddharthab,

It looks like this pull-request is has been made against the siddharthab/rnaseq master branch.
The master branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to master are only allowed if they come from the siddharthab/rnaseq dev branch.

You do not need to close this PR, you can change the target branch to dev by clicking the "Edit" button at the top of this page.
Note that even after this, the test will continue to show as failing until you push a new commit.

Thanks again for your contribution!

@siddharthab siddharthab closed this Sep 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Clarify infer strandedness from current subsampling + infer step
7 participants