Migrate component create to nf test

.github/workflows/lint-code.yml

@@ -92,8 +92,29 @@ jobs:
       - uses: actions/checkout@v2
       - uses: actions/setup-python@v3
         with:
-          python-version: "3.11"
-      - run: pip install mypy types-PyYAML
+          python-version: 3.11
+          cache: "pip"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip -r requirements-dev.txt
+          pip install -e .
+
+      - name: Cache nf-test installation
+        id: cache-software
+        uses: actions/cache@v3
+        with:
+          path: |
+            /usr/local/bin/nf-test
+            /home/runner/.nf-test/nf-test.jar
+          key: ${{ runner.os }}-${{ env.NFTEST_VER }}-nftest
+
+      - name: Install nf-test
+        if: steps.cache-software.outputs.cache-hit != 'true'
+        run: |
+          wget -qO- https://code.askimed.com/install/nf-test | bash
+          sudo mv nf-test /usr/local/bin/
+
       - name: Get Python changed files
         id: changed-py-files
         uses: tj-actions/changed-files@v23

.github/workflows/pytest.yml

@@ -64,6 +64,21 @@ jobs:
         with:
           version: "latest-everything"
 
+      - name: Cache nf-test installation
+        id: cache-software
+        uses: actions/cache@v3
+        with:
+          path: |
+            /usr/local/bin/nf-test
+            /home/runner/.nf-test/nf-test.jar
+          key: ${{ runner.os }}-${{ env.NFTEST_VER }}-nftest
+
+      - name: Install nf-test
+        if: steps.cache-software.outputs.cache-hit != 'true'
+        run: |
+          wget -qO- https://code.askimed.com/install/nf-test | bash
+          sudo mv nf-test /usr/local/bin/
+
       - name: Test with pytest
         run: python3 -m pytest tests/ --color=yes --cov-report=xml --cov-config=.github/.coveragerc --cov=nf_core
 

.prettierignore

@@ -4,4 +4,5 @@ slackreport.json
 docs/api/_build
 testing
 nf_core/module-template/modules/meta.yml
-nf_core/module-template/tests/test.yml
+nf_core/module-template/modules/tests/tags.yml
+nf_core/subworkflow-template/subworkflows/tests/tags.yml

CHANGELOG.md

@@ -27,6 +27,7 @@
 
 ### General
 
+- Change testing framework for modules and subworkflows from pytest to nf-test ([#2490](https://github.com/nf-core/tools/pull/2490))
 - `bump_version` keeps now the indentation level of the updated version entries ([#2514](https://github.com/nf-core/tools/pull/2514))
 
 # [v2.10 - Nickel Ostrich](https://github.com/nf-core/tools/releases/tag/2.10) + [2023-09-25]

Dockerfile

@@ -31,6 +31,11 @@ ENV NXF_VER ${NXF_VER}
 RUN curl -s https://get.nextflow.io | bash \
     && mv nextflow /usr/local/bin \
     && chmod a+rx /usr/local/bin/nextflow
+# Install nf-test
+RUN curl -fsSL https://code.askimed.com/install/nf-test | bash \
+    && mv nf-test /usr/local/bin \
+    && chmod a+rx /usr/local/bin/nf-test
+
 # Add the nf-core source files to the image
 COPY . /usr/src/nf_core
 WORKDIR /usr/src/nf_core

README.md

@@ -39,7 +39,6 @@ A python package with helper tools for the nf-core community.
   - [`modules remove` - Remove a module from a pipeline](#remove-a-module-from-a-pipeline)
   - [`modules patch` - Create a patch file for a module](#create-a-patch-file-for-a-module)
   - [`modules create` - Create a module from the template](#create-a-new-module)
-  - [`modules create-test-yml` - Create the `test.yml` file for a module](#create-a-module-test-config-file)
   - [`modules lint` - Check a module against nf-core guidelines](#check-a-module-against-nf-core-guidelines)
   - [`modules test` - Run the tests for a module](#run-the-tests-for-a-module-using-pytest)
   - [`modules bump-versions` - Bump software versions of modules](#bump-bioconda-and-container-versions-of-modules-in)
@@ -53,7 +52,6 @@ A python package with helper tools for the nf-core community.
   - [`subworkflows update` - Update subworkflows in a pipeline](#update-subworkflows-in-a-pipeline)
   - [`subworkflows remove` - Remove a subworkflow from a pipeline](#remove-a-subworkflow-from-a-pipeline)
   - [`subworkflows create` - Create a subworkflow from the template](#create-a-new-subworkflow)
-  - [`subworkflows create-test-yml` - Create the `test.yml` file for a subworkflow](#create-a-subworkflow-test-config-file)
   - [`subworkflows lint` - Check a subworkflow against nf-core guidelines](#check-a-subworkflow-against-nf-core-guidelines)
   - [`subworkflows test` - Run the tests for a subworkflow](#run-the-tests-for-a-subworkflow-using-pytest)
 - [Citation](#citation)
@@ -947,20 +945,6 @@ fake_command: nf-core modules create fastqc --author @nf-core-bot  --label proce
 
 ![`cd modules && nf-core modules create fastqc --author @nf-core-bot  --label process_low --meta --force`](docs/images/nf-core-modules-create.svg)
 
-### Create a module test config file
-
-All modules on [nf-core/modules](https://github.com/nf-core/modules) have a strict requirement of being unit tested using minimal test data.
-To help developers build new modules, the `nf-core modules create-test-yml` command automates the creation of the yaml file required to document the output file `md5sum` and other information generated by the testing.
-After you have written a minimal Nextflow script to test your module `tests/modules/<tool>/<subtool>/main.nf`, this command will run the tests for you and create the `tests/modules/<tool>/<subtool>/test.yml` file.
-
-<!-- RICH-CODEX
-working_dir: tmp/modules
-extra_env:
-  PROFILE: 'conda'
--->
-
-![`nf-core modules create-test-yml fastqc --no-prompts --force`](docs/images/nf-core-modules-create-test.svg)
-
 ### Check a module against nf-core guidelines
 
 Run the `nf-core modules lint` command to check modules in the current working directory (pipeline or nf-core/modules clone) against nf-core guidelines.
@@ -974,15 +958,12 @@ before_command: sed 's/1.13a/1.10/g' modules/multiqc/main.nf > modules/multiqc/m
 
 ![`nf-core modules lint multiqc`](docs/images/nf-core-modules-lint.svg)
 
-### Run the tests for a module using pytest
-
-To run unit tests of a module that you have installed or the test created by the command [`nf-core modules create-test-yml`](#create-a-module-test-config-file), you can use `nf-core modules test` command. This command runs the tests specified in `modules/tests/software/<tool>/<subtool>/test.yml` file using [pytest](https://pytest-workflow.readthedocs.io/en/stable/).
+### Create a test for a module
 
-:::info
-This command uses the pytest argument `--git-aware` to avoid copying the whole `.git` directory and files ignored by `git`. This means that it will only include files listed by `git ls-files`. Remember to **commit your changes** after adding a new module to add the new files to your git index.
-:::
+All modules on [nf-core/modules](https://github.com/nf-core/modules) have a strict requirement of being unit tested using minimal test data. We use [nf-test](https://code.askimed.com/nf-test/) as our testing framework.
+Each module comes already with a template for the test file in `test/main.nf.test`. Replace the placeholder code in that file with your specific input, output and proces. In order to generate the corresponding snapshot after writing your test, you can use the `nf-core modules test` command. This command will run `nf-test test` twice, to also check for snapshot stability, i.e. that the same snapshot is generated on multiple runs.
 
-You can specify the module name in the form TOOL/SUBTOOL in command line or provide it later by prompts.
+You can specify the module name in the form TOOL/SUBTOOL in the command or provide it later through interactive prompts.
 
 <!-- RICH-CODEX
 working_dir: tmp/modules
@@ -991,7 +972,15 @@ extra_env:
   PROFILE: 'conda'
 -->
 
-![`nf-core modules test samtools/view --no-prompts`](docs/images/nf-core-modules-test.svg)
+![`nf-core modules test fastqc --no-prompts --force`](docs/images/nf-core-modules-test.svg)
+
+In case you changed something in the test and want to update the snapshot, run
+
+```bash
+nf-core modules test --update
+```
+
+If you want to run the test only once without checking for snapshot stability, you can use the `--once` flag.
 
 ### Bump bioconda and container versions of modules in
 
@@ -1221,19 +1210,29 @@ fake_command: nf-core subworkflows create bam_stats_samtools --author @nf-core-b
 
 ![`nf-core subworkflows create bam_stats_samtools --author @nf-core-bot --force`](docs/images/nf-core-subworkflows-create.svg)
 
-### Create a subworkflow test config file
+### Create a test for a subworkflow
+
+All subworkflows on [nf-core/modules](https://github.com/nf-core/modules) have a strict requirement of being unit tested using minimal test data. We use [nf-test](https://code.askimed.com/nf-test/) as our testing framework.
+Each subworkflow comes already with a template for the test file in `test/main.nf.test`. Replace the placeholder code in that file with your specific input, output and proces. In order to generate the corresponding snapshot after writing your test, you can use the `nf-core subworkflows test` command. This command will run `nf-test test` twice, to also check for snapshot stability, i.e. that the same snapshot is generated on multiple runs.
 
-All subworkflows on [nf-core/modules](https://github.com/nf-core/modules) have a strict requirement of being unit tested using minimal test data.
-To help developers build new subworkflows, the `nf-core subworkflows create-test-yml` command automates the creation of the yaml file required to document the output file `md5sum` and other information generated by the testing.
-After you have written a minimal Nextflow script to test your subworkflow in `/tests/subworkflow/<subworkflow_name>/main.nf`, this command will run the tests for you and create the `/tests/subworkflow/<tool>/<subtool>/test.yml` file.
+You can specify the subworkflow name in the command or provide it later through interactive prompts.
 
 <!-- RICH-CODEX
 working_dir: tmp/modules
+timeout: 30
 extra_env:
   PROFILE: 'conda'
 -->
 
-![`nf-core subworkflows create-test-yml bam_stats_samtools --no-prompts --force`](docs/images/nf-core-subworkflows-create-test.svg)
+![`nf-core subworkflows test bam_rseqc --no-prompts`](docs/images/nf-core-subworkflows-test.svg)
+
+In case you changed something in the test and want to update the snapshot, run
+
+```bash
+nf-core subworkflows test --update
+```
+
+If you want to run the test only once without checking for snapshot stability, you can use the `--once` flag.
 
 ### Check a subworkflow against nf-core guidelines
 
@@ -1249,25 +1248,6 @@ extra_env:
 
 ![`nf-core subworkflows lint bam_stats_samtools`](docs/images/nf-core-subworkflows-lint.svg)
 
-### Run the tests for a subworkflow using pytest
-
-To run unit tests of a subworkflow that you have installed or the test created by the command [`nf-core subworkflow create-test-yml`](#create-a-subworkflow-test-config-file), you can use `nf-core subworkflows test` command. This command runs the tests specified in `tests/subworkflows/<subworkflow_name>/test.yml` file using [pytest](https://pytest-workflow.readthedocs.io/en/stable/).
-
-:::info
-This command uses the pytest argument `--git-aware` to avoid copying the whole `.git` directory and files ignored by `git`. This means that it will only include files listed by `git ls-files`. Remember to **commit your changes** after adding a new subworkflow to add the new files to your git index.
-:::
-
-You can specify the subworkflow name in the form TOOL/SUBTOOL in command line or provide it later by prompts.
-
-<!-- RICH-CODEX
-working_dir: tmp/modules
-timeout: 30
-extra_env:
-  PROFILE: 'conda'
--->
-
-![`nf-core subworkflows test bam_rseqc --no-prompts`](docs/images/nf-core-subworkflows-test.svg)
-
 ## Citation
 
 If you use `nf-core tools` in your work, please cite the `nf-core` publication as follows: