Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated config for running longranger with and without LSF #19

Merged
merged 1 commit into from
Sep 11, 2023
Merged

Updated config for running longranger with and without LSF #19

merged 1 commit into from
Sep 11, 2023

Conversation

muffato
Copy link
Member

@muffato muffato commented Sep 8, 2023

Hi @ksenia-krasheninnikova and @priyanka-surana

I've run a few tests for lonranger and found some things that can be removed / tidied up, and also, as per Priyanka's comments, some things that could be improved to make the pipeline less Sanger-specific.

First of all, we should remember that the Longranger container cannot be made public. The modules modules/local/longranger/*/main.nf currently point at our GitLab and we make them use a private container on GitHub for GitHub CI (the if(System.getenv('GITHUB_ACTION') != null ) statements in conf/modules.config).
If anyone else wants to run Longranger, they will have to override the container definitions.

Then, by default Longranger will run in "local" mode. To make it work over LSF, it needs some extra configuration.
The envWhitelist and runOptions directives exist in the "singularity" scope and only the lines that were in nextflow.config could actually work. The lines in conf/modules.config couldn't work because they were inside a process { ... } block.

I moved all the Sanger LSF config to a different profile in conf/longranger_lsf_sanger.config that has to be defined at the end of conf/modules.config so that it can override the non-LSF settings.
To enable it, just add longranger_lsf_sanger to the list of profiles alongside test and singularity.

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the sanger-tol/genomeassembly branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@muffato muffato mentioned this pull request Sep 8, 2023
Merged
10 tasks
@github-actions
Copy link

github-actions bot commented Sep 8, 2023 •
edited
Loading

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit dfafb26

+| ✅ 131 tests passed       |+
#| ❔  18 tests were ignored |#
!| ❗  15 tests had warnings |!

❗ Test warnings:

  • files_exist - File not found: conf/igenomes.config
  • readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your prefered methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in WorkflowMain.groovy: Add Zenodo DOI for pipeline after first release
  • pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
  • pipeline_todos - TODO string in output.md: Write this documentation describing your workflow's output
  • pipeline_todos - TODO string in genomeassembly.nf: Add all file path parameters for the pipeline to the list below
  • pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
  • pipeline_todos - TODO string in ci.yml: You can customise CI pipeline run tests as required
  • system_exit - System.exit in WorkflowSanger-tol-genomeassembly.groovy: System.exit(1) [line 51]
  • schema_description - No description provided in schema for parameter: timestamp
  • schema_description - No description provided in schema for parameter: hifiasm
  • schema_description - No description provided in schema for parameter: hifiasmhic

❔ Tests ignored:

  • files_exist - File is ignored: assets/nf-core-genomeassembly_logo_light.png
  • files_exist - File is ignored: docs/images/nf-core-genomeassembly_logo_light.png
  • files_exist - File is ignored: docs/images/nf-core-genomeassembly_logo_dark.png
  • files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
  • files_exist - File is ignored: .github/workflows/awstest.yml
  • files_exist - File is ignored: .github/workflows/awsfulltest.yml
  • nextflow_config - Config variable ignored: manifest.name
  • nextflow_config - Config variable ignored: manifest.homePage
  • files_unchanged - File ignored due to lint config: LICENSE or LICENSE.md or LICENCE or LICENCE.md
  • files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
  • files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
  • files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
  • files_unchanged - File ignored due to lint config: assets/sendmail_template.txt
  • files_unchanged - File does not exist: assets/nf-core-genomeassembly_logo_light.png
  • files_unchanged - File does not exist: docs/images/nf-core-genomeassembly_logo_light.png
  • files_unchanged - File does not exist: docs/images/nf-core-genomeassembly_logo_dark.png
  • files_unchanged - File ignored due to lint config: lib/NfcoreTemplate.groovy
  • files_unchanged - File ignored due to lint config: .gitignore or .prettierignore or pyproject.toml

✅ Tests passed:

Run details

  • nf-core/tools version 2.8
  • Run at 2023-09-08 15:30:58

@ksenia-krasheninnikova ksenia-krasheninnikova merged commit c6ce6fb into template_merge Sep 11, 2023
4 of 6 checks passed
@ksenia-krasheninnikova
Copy link
Contributor

This code works fine for me in the local mode. The issue with LSF mode is still there, longranger doesn't run properly in LSF mode for me, but ignoring it for now.

@muffato muffato deleted the longranger_lsf branch September 11, 2023 15:25
@ksenia-krasheninnikova
Copy link
Contributor

Issue was resolved. Works fine in LSF mode with singularity 3.9.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ksenia-krasheninnikova ksenia-krasheninnikova Awaiting requested review from ksenia-krasheninnikova

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@muffato @ksenia-krasheninnikova