Merge pull request #81 from LUMC/release_1.2.1

Release 1.2.1

.github/release_checklist.md

@@ -16,6 +16,6 @@ from history.rst.
 - [ ] Push tested packages to pypi
 - [ ] merge `master` branch back into `develop`.
 - [ ] Add updated version number to develop
-- [ ] Update the package on bioconda
+- [ ] Update the package on conda-forge
 - [ ] Build the new tag on readthedocs. Only build the last patch version of
-each minor version. So `1.1.1` and `1.2.0` but not `1.1.0`, `1.1.1` and `1.2.0`.
+each minor version. So `1.1.1` and `1.2.0` but not `1.1.0`, `1.1.1` and `1.2.0`.

HISTORY.rst

@@ -7,6 +7,17 @@ Changelog
 .. NOTE: This document is user facing. Please word the changes in such a way
 .. that users understand how the changes affect the new version.
 
+version 1.2.1
+---------------------------
++ Since pytest 4.5.0 unknown markers give a warning. ``@pytest.mark.workflow``
+  markers are now added to the configuration. Information on usage shows up
+  with ``pytest --mark``.
++ Updated documentation to reflect the move to conda-forge as requested on
+  `this github issue
+  <https://github.com/bioconda/bioconda-recipes/issues/13964>`_.
++ Updated documentation on how to test Cromwell + WDL pipelines.
+
+
 version 1.2.0
 ---------------------------
 + Giving a ``--basetemp`` directory that is within pytest's current working

README.rst

@@ -8,8 +8,8 @@ pytest-workflow
   :target: https://pypi.org/project/pytest-workflow/
   :alt:
 
-.. image:: https://img.shields.io/conda/v/bioconda/pytest-workflow.svg
-  :target: https://bioconda.github.io/recipes/pytest-workflow/README.html
+.. image:: https://img.shields.io/conda/v/conda-forge/pytest-workflow.svg
+  :target: https://anaconda.org/conda-forge/pytest-workflow
   :alt:
 
 .. image:: https://img.shields.io/pypi/pyversions/pytest-workflow.svg
@@ -20,10 +20,6 @@ pytest-workflow
   :target: https://github.com/LUMC/pytest-workflow/blob/master/LICENSE
   :alt:
 
-.. image:: https://api.codacy.com/project/badge/Grade/f8bc14b0a507429eac7c06194fafcd59
-  :target: https://www.codacy.com/app/LUMC/pytest-workflow?utm_source=github.com&utm_medium=referral&utm_content=LUMC/pytest-workflow&utm_campaign=Badge_Grade
-  :alt:
-
 .. image:: https://travis-ci.org/LUMC/pytest-workflow.svg?branch=develop
   :target: https://travis-ci.org/LUMC/pytest-workflow
   :alt:
@@ -49,11 +45,14 @@ and 3.7. Python 2 is not supported.
 - Create a ``tests`` directory in the root of your repository.
 - Create your test yaml files in the ``tests`` directory.
 
-Pytest-workflow is also available as a `conda package on bioconda
-<https://bioconda.github.io/recipes/pytest-workflow/README.html>`_. Follow
-`these instructions <https://bioconda.github.io/index.html#set-up-channels>`_
-to set up channels properly in order to use bioconda. After that ``conda
-install pytest-workflow`` can be used to install pytest-workflow.
+Pytest-workflow is also available as a `conda package on conda-forge
+<https://anaconda.org/conda-forge/pytest-workflow>`_. Follow
+`these instructions 
+<http://conda-forge.org/docs/user/introduction.html#how-can-i-install-packages-from-conda-forge>`_ 
+to set up channels properly in order to use conda-forge. Alternatively,
+you can `set up the channels correctly for use with bioconda 
+<https://bioconda.github.io/index.html#set-up-channels>`_. After that ``conda
+install pytest-workflow`` can be used to install pytest-workflow. 
 
 Quickstart
 ==========

docs/examples.rst

@@ -26,37 +26,47 @@ WDL with Cromwell example
 Below an example yaml file is explained which can be used to test a WDL
 pipeline run through Cromwell.
 
-One problem with Cromwell is the way it handles relative paths and how it
-handles the input file:
+By default Cromwell outputs its files in the execution folder in a
+deeply-nested folder structure. Cromwell can output to a separate
+workflow-outputs folder and since Cromwell version 40 it can also output the
+files in a structure that is not nested. For more information check the
+`Cromwell documentation on global workflow options
+<https://cromwell.readthedocs.io/en/stable/wf_options/Overview/#global-workflow-options>`_.
 
-+ Relative paths are written only within the ``cromwell-executions`` folder.
-  If you want to write outside this folder you need absolute paths. This is
-  fine but for testing your pipeline ``pytest-workflow`` creates a temporary
-  folder from which the pipeline is run. You don't know beforehand which path
-  this is, but you could use the environment variable ``$PWD``.
-+ However the second problem is that inputs can only be supplied to Cromwell in
-  a json file, not on the command line. So you cannot dynamically choose an
-  output folder. You have to rewrite the input file.
+In order to run Cromwell for CI tests an options file should be present in the
+repository with the following contents:
 
-To fix this problem you can write ``command`` to be a bash script that injects
-``$PWD`` into the inputs.json.
+.. code-block:: json
+
+    {
+    "final_workflow_outputs_dir": "test-output",
+    "use_relative_output_paths": true,
+    "default_runtime_attributes": {
+      "docker_user": "$EUID"
+      }
+    }
+
+``final_workflow_outputs_dir`` will make sure all the files produced in the
+workflow will be copied to the ``final_workflow_outputs_dir``.
+``use_relative_output_paths`` will get rid of all the Cromwell specific folders
+such as ``call-myTask`` etc. The ``default_runtime_attributes`` are only
+necessary when using docker containers. It will make sure all the files are
+created by the same user that runs the test (docker containers run as root by
+default). This will ensure that files can be deleted by pytest-workflow
+afterwards.
+
+The following yaml file tests a Cromwell pipeline. In this case Cromwell is
+installed via conda. The conda installation adds a wrapper to Cromwell so it
+can be used as a command, instead of having to use the jar.
 
 .. code-block:: yaml
 
   - name: My pipeline
-    command: >-
-      bash -c '
-      TEST_JSON=tests/inputs/my_pipeline_test1.json ;
-      sed -i "2i\"my_pipeline.output_dir\":\"$PWD/test-output\"," $TEST_JSON ;
-      cromwell run -i $TEST_JSON simple.wdl'
+    command: cromwell run -i inputs.json -o options.json moo.wdl
     files:
       - path: test-output/moo.txt.gz
         md5sum: 173fd8023240a8016033b33f42db14a2
     stdout:
       contains:
         - "WorkflowSucceededState"
 
-``sed -i "2i\"my_pipeline.output_dir\":\"$PWD/test-output\"," $TEST_JSON``
-inserts ``"my_pipeline.output_dir":"</pytest/temporary/dir>/test-output",`` on
-the second line of ``$TEST_JSON``. This solves the problem. File paths can now
-be traced from ``test-output`` as demonstrated in the example.

docs/installation.rst

@@ -4,13 +4,15 @@ Installation
 
 Pytest-workflow is tested on python 3.5, 3.6 and 3.7. Python 2 is not supported.
 
-In a virtual environment:
+In a virtual environment
+------------------------
 
 - Create a new python3 virtual environment.
 - Make sure your virtual environment is activated.
 - Install using pip ``pip install pytest-workflow``
 
-On Ubuntu or Debian:
+On Ubuntu or Debian
+-------------------
 
 - This requires the ``python3`` and ``python3-pip`` packages to be installed.
 - Installing
@@ -27,10 +29,17 @@ On Ubuntu or Debian:
     the python2 version of ``pytest``. This is because python2 is the default
     python on any distribution released before January 1st 2020.
 
-Pytest-workflow is also available as a `conda package on bioconda
-<https://bioconda.github.io/recipes/pytest-workflow/README.html>`_.
+Conda
+-----
+
+Pytest-workflow is also available as a `conda package on conda-forge
+<https://anaconda.org/conda-forge/pytest-workflow>`_.
 To install with conda:
 
-- `Set up conda to use the bioconda channel
-  <https://bioconda.github.io/index.html#set-up-channels>`_
+- `Set up conda to use the conda-forge channel
+  <http://conda-forge.org/docs/user/introduction.html#how-can-i-install-packages-from-conda-forge>`_
+
+  - If you want to use pytest-workflow together with bioconda you can follow
+    `the instructions here
+    <https://bioconda.github.io/index.html#set-up-channels>`_.
 - ``conda install pytest-workflow``

setup.py

@@ -21,7 +21,7 @@
 
 setup(
     name="pytest-workflow",
-    version="1.2.0",
+    version="1.2.1",
     description="A pytest plugin for configuring workflow/pipeline tests "
                 "using YAML files",
     author="Leiden University Medical Center",

src/pytest_workflow/plugin.py

@@ -96,6 +96,16 @@ def pytest_collect_file(path, parent):
 
 def pytest_configure(config: PytestConfig):
     """This runs before tests start and adds values to the config."""
+
+    #  Add marker to the config to prevent issues caused by:
+    #  https://github.com/pytest-dev/pytest/issues/4826
+    # Errors are now emitted when unknown marks are included
+    config.addinivalue_line(
+        "markers",
+        "workflow(name): mark test to run only with the given "
+        "workflow name. Also provides access to the workflow_dir "
+        "fixture."
+    )
     # We need to add a workflow queue to some central variable. Instead of
     # using a global variable we add a value to the config.
     # Using setattr is not the nicest way of doing things, but having something

tests/functional/simple_wdl_test_cases.yml

@@ -1,12 +1,38 @@
 - name: simple wdl
   command: >-
-    bash -c '
-    TEST_JSON=simple.json ;
-    sed -i "2i\"string_to_zip.out_path\":\"$PWD/moo.txt.gz\"," $TEST_JSON ;
-    cromwell run -i $TEST_JSON simple.wdl'
+     cromwell run -i simple.json -o simple.options.json simple.wdl
   files:
-    - path: moo.txt.gz
-      md5sum: 173fd8023240a8016033b33f42db14a2
+    - path: test-output/rand/0.bin
+    - path: test-output/rand/1.bin
+    - path: test-output/rand/2.bin
+    - path: test-output/rand/3.bin
+    - path: test-output/rand/4.bin
+    - path: test-output/rand/5.bin
+    - path: test-output/rand/6.bin
+    - path: test-output/rand/7.bin
+    - path: test-output/rand/8.bin
+    - path: test-output/rand/9.bin
+    - path: test-output/b64/0.txt
+    - path: test-output/b64/1.txt
+    - path: test-output/b64/2.txt
+    - path: test-output/b64/3.txt
+    - path: test-output/b64/4.txt
+    - path: test-output/b64/5.txt
+    - path: test-output/b64/6.txt
+    - path: test-output/b64/7.txt
+    - path: test-output/b64/8.txt
+    - path: test-output/b64/9.txt
+    - path: test-output/randgz/0.txt.gz
+    - path: test-output/randgz/1.txt.gz
+    - path: test-output/randgz/2.txt.gz
+    - path: test-output/randgz/3.txt.gz
+    - path: test-output/randgz/4.txt.gz
+    - path: test-output/randgz/5.txt.gz
+    - path: test-output/randgz/6.txt.gz
+    - path: test-output/randgz/7.txt.gz
+    - path: test-output/randgz/8.txt.gz
+    - path: test-output/randgz/9.txt.gz
+    - path: test-output/all_data.gz
   stdout:
     contains:
       - "WorkflowSucceededState"

tests/functional/test_cromwell.py

@@ -41,12 +41,21 @@ def simple_wdl_json():
         return json_handle.read()
 
 
+@pytest.fixture
+def simple_wdl_options_json():
+    json_path = (Path(__file__).parent.parent / Path("pipelines") /
+                 Path("wdl") / Path("simple.options.json"))
+    with json_path.open("r") as json_handle:
+        return json_handle.read()
+
+
 @pytest.mark.functional
 def test_cromwell(testdir, simple_wdl_yaml, simple_wdl_contents,
-                  simple_wdl_json):
+                  simple_wdl_json, simple_wdl_options_json):
     testdir.makefile(ext=".json", simple=simple_wdl_json)
     testdir.makefile(ext=".wdl", simple=simple_wdl_contents)
     testdir.makefile(ext=".yml", test_cromwell=simple_wdl_yaml)
+    testdir.makefile(ext=".options.json", simple=simple_wdl_options_json)
     result = testdir.runpytest("-v")
     exit_code = result.ret
     assert exit_code == 0

tests/pipelines/wdl/simple.json

@@ -1,3 +1,3 @@
 {
-  "string_to_zip.string": "moo"
+  "random_zip.lines_to_read": 10
 }

tests/pipelines/wdl/simple.options.json

@@ -0,0 +1,7 @@
+{
+  "final_workflow_outputs_dir": "test-output",
+  "use_relative_output_paths": true,
+  "default_runtime_attributes": {
+    "docker_user": "$EUID"
+  }
+}