Merge pull request #2 from nf-core/dev

Get latest dev version

Author: jfy133

.github/RELEASE_CHECKLIST.md

@@ -0,0 +1,18 @@
+## Before release
+
+1. Check issue milestones to see outstanding issues to resolve if possible or transfer to the milestones for the next release e.g. [`v1.9`](https://github.com/nf-core/tools/issues?q=is%3Aopen+is%3Aissue+milestone%3A1.9)
+2. Create a PR to `dev` to bump the version in  `CHANGELOG.md` and `setup.py`.
+3. Make sure all CI tests are passing!
+4. Create a PR from `dev` to `master`
+5. Make sure all CI tests are passing again (additional tests are run on PRs to `master`)
+6. Request review (2 approvals required)
+7. Most importantly, pick an undeniably outstanding [name](http://www.codenamegenerator.com/) for the release where *Prefix* = *Metal* and *Dictionary* = *Animal*.
+8. Merge the PR into `master`
+9. Once CI tests on commit have passed, create a new release copying the `CHANGELOG` for that release into the description section.
+
+## After release
+
+1. Check the automated template synchronisation has been triggered properly. This should automatically open PRs directly to individual pipeline repos with the appropriate changes to update the pipeline template.
+2. Check that the automatic `PyPi` deployment has worked: [pypi.org/project/nf-core](https://pypi.org/project/nf-core/)
+3. Check `BioConda` has an automated PR to bump the version, and merge. eg. [bioconda/bioconda-recipes #20065](https://github.com/bioconda/bioconda-recipes/pull/20065)
+4. Create a tools PR to `dev` to bump back to the next development version in  `CHANGELOG.md` and `setup.py`

.github/workflows/branch.yml

@@ -0,0 +1,35 @@
+name: nf-core branch protection
+# This workflow is triggered on PRs to master branch on the repository
+# It fails when someone tries to make a PR against the nf-core `master` branch instead of `dev`
+on:
+  pull_request:
+    branches: [master]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+
+      # PRs to the nf-core repo master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
+      - name: Check PRs
+        if: github.repository == 'nf-core/tools'
+        run: |
+          { [[ ${{github.event.pull_request.head.repo.full_name}} == nf-core/tools ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
+
+      # If the above check failed, post a comment on the PR explaining the failure
+      - name: Post PR comment
+        if: failure()
+        uses: mshick/add-pr-comment@v1
+        with:
+          message: |
+            Hi @${{ github.event.pull_request.user.login }},
+
+            It looks like this pull-request is has been made against the ${{github.event.pull_request.head.repo.full_name}} `master` branch.
+            The `master` branch on nf-core repositories should always contain code from the latest release.
+            Beacuse of this, PRs to `master` are only allowed if they come from the ${{github.event.pull_request.head.repo.full_name}} `dev` branch.
+
+            You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page.
+
+            Thanks again for your contribution!
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          allow-repeats: false

.github/workflows/code-tests.yml

@@ -2,7 +2,6 @@ name: Python tests
 # This workflow is triggered on pushes and PRs to the repository.
 on: [push, pull_request]
 
-
 jobs:
   PythonLint:
 
@@ -14,29 +13,35 @@ jobs:
     steps:
     - uses: actions/checkout@v2
       name: Check out source-code repository
+
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v1
       with:
         python-version: ${{ matrix.python-version }}
+
     - name: Install python dependencies
       run: |
         python -m pip install --upgrade pip pytest pytest-datafiles pytest-cov mock jsonschema
         pip install .
+
     - name: Install Nextflow
       run: |
         mkdir /tmp/nextflow
         cd /tmp/nextflow
         wget -qO- get.nextflow.io | bash
         sudo ln -s /tmp/nextflow/nextflow /usr/local/bin/nextflow
+
     - name: Lint with flake8
       run: |
         pip install flake8
         # stop the build if there are Python syntax errors or undefined names
         flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
         # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
         flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
+
     - name: Test with pytest
       run: python3 -m pytest --color=yes --cov-report=xml --cov-config=.github/.coveragerc --cov=nf_core
+
     - uses: codecov/codecov-action@v1
       name: Upload code coverage report
       with:

.github/workflows/create-lint-wf.yml

@@ -6,20 +6,25 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v2
+        name: Check out source-code repository
+
       - name: Set up Python 3.7
         uses: actions/setup-python@v1
         with:
           python-version: 3.7
+
       - name: Install python dependencies
         run: |
           python -m pip install --upgrade pip
           pip install .
+
       - name: Install Nextflow
         run: |
           mkdir /tmp/nextflow
           cd /tmp/nextflow
           wget -qO- get.nextflow.io | bash
           sudo ln -s /tmp/nextflow/nextflow /usr/local/bin/nextflow
+
       - name: Run nf-core tools
         run: |
           nf-core create -n testpipeline -d "This pipeline is for testing" -a "Testing McTestface"

.github/workflows/deploy-pypi.yml

@@ -1,10 +1,11 @@
 name: Publish nf-core to PyPI
-on: push
+on:
+  release:
+    types: [published]
 
 jobs:
   build-n-publish:
-    name: Build and publish nf-core to PyPI
-    runs-on: ubuntu-18.04
+    runs-on: ubuntu-latest
 
     steps:
       - uses: actions/checkout@v2
@@ -24,7 +25,7 @@ jobs:
         run: python setup.py sdist bdist_wheel
 
       - name: Publish dist to PyPI
-        if: github.repository == 'nf-core/tools' && startsWith(github.event.ref, 'refs/tags')
+        if: github.repository == 'nf-core/tools'
         uses: pypa/gh-action-pypi-publish@master
         with:
           password: ${{ secrets.pypi_password }}

.github/workflows/markdown-lint.yml

@@ -6,10 +6,13 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v2
+
       - uses: actions/setup-node@v1
         with:
           node-version: '10'
+
       - name: Install markdownlint
         run: npm install -g markdownlint-cli
+
       - name: Run Markdownlint
         run: markdownlint ${GITHUB_WORKSPACE} -c ${GITHUB_WORKSPACE}/.github/markdownlint.yml

.github/workflows/sync.yml

@@ -0,0 +1,41 @@
+name: Sync template
+on:
+  release:
+    types: [published]
+
+jobs:
+  sync-all:
+    name: Sync all pipelines
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v2
+        name: Check out source-code repository
+
+      - name: Set up Python 3.7
+        uses: actions/setup-python@v1
+        with:
+          python-version: 3.7
+
+      - name: Install python dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install .
+
+      - name: Install Nextflow
+        run: |
+          mkdir /tmp/nextflow
+          cd /tmp/nextflow
+          wget -qO- get.nextflow.io | bash
+          sudo ln -s /tmp/nextflow/nextflow /usr/local/bin/nextflow
+
+      - name: Configure git
+        run: |
+          git config user.email "[email protected]"
+          git config user.name "nf-core-bot"
+
+      - name: Run synchronisation
+        if: github.repository == 'nf-core/tools'
+        env:
+          AUTH_TOKEN: ${{ secrets.nf_core_bot_auth_token }}
+        run: nf-core sync --all --username nf-core-bot --auth-token $AUTH_TOKEN

.github/workflows/tools-api-docs.yml

@@ -37,5 +37,5 @@ jobs:
 
           $GITHUB_REF - $GITHUB_SHA
           "
-          git remote add nf-core "https://${NF_CORE_BOT}@github.com/nf-core/tools.git" > /dev/null 2>&1
-          git push --force --quiet --set-upstream nf-core api-doc
+          git remote add nf-core https://github.com/nf-core/tools.git
+          git push --force --set-upstream nf-core api-doc

.travis.yml

@@ -1,13 +1,27 @@
 # nf-core/tools: Changelog
 
-## v1.9dev
+## v1.10dev
+
+### Linting
+
+* Refactored PR branch tests to be a little clearer.
+* Linting error docs explain how to add an additional branch protecton rule to the `branch.yml` GitHub Actions workflow.
+* Adapted linting docs to the new PR branch tests.
+* Added test for template `{{ cookiecutter.var }}` placeholders
+
+### Other
+
+* Added CI test to check for PRs against `master` in tools repo
+* CI PR branch tests fixed & now automatically add a comment on the PR if failing, explaining what is wrong
+
+## v1.9
 
 ### Continuous integration
 
-* Travis CI tests are now deprecated within the pipeline template. Please switch to GitHub Actions.
+* Travis CI tests are now deprecated in favor of GitHub Actions within the pipeline template.
   * `nf-core bump-version` support has been removed for `.travis.yml`
   * `nf-core lint` now fails if a `.travis.yml` file is found
-* Ported nf-core/tools Travis CI automation to GitHub Actions
+* Ported nf-core/tools Travis CI automation to GitHub Actions.
 * Fixed the build for the nf-core/tools API documentation on the website
 
 ### Template
@@ -16,28 +30,31 @@
 * Removed the requirement for R in the conda environment
 * Make `params.multiqc_config` give an _additional_ MultiQC config file instead of replacing the one that ships with the pipeline
 * Ignore only `tests/` and `testing/` directories in `.gitignore` to avoid ignoring `test.config` configuration file
+* Rephrase docs to promote usage of containers over Conda to ensure reproducibility
+* Stage the workflow summary YAML file within MultiQC work directory
 
 ### Linting
 
 * Removed linting for CircleCI
+* Allow any one of `params.reads` or `params.input` or `params.design` before warning
 * Added whitespace padding to lint error URLs
 * Improved documentation for lint errors
 * Allow either `>=` or `!>=` in nextflow version checks (the latter exits with an error instead of just warning) [#506](https://github.com/nf-core/tools/issues/506)
 * Check that `manifest.version` ends in `dev` and throw a warning if not
   * If running with `--release` check the opposite and fail if not
 * Tidied up error messages and syntax for linting GitHub actions branch tests
 * Add YAML validator
+* Don't print test results if we have a critical error
 
 ### Other
 
+* Fix automatic synchronisation of the template after releases of nf-core/tools
 * Improve documentation for installing `nf-core/tools`
-* Use `stderr` instead of `stdout` for header artwork
 * Replace preprint by the new nf-core publication in Nature Biotechnology :champagne:
+* Use `stderr` instead of `stdout` for header artwork
 * Tolerate unexpected output from `nextflow config` command
-
-### Template pipeline
-
-* Stage the workflow summary YAML file within MultiQC work directory
+* Add social preview image
+* Added a [release checklist](.github/RELEASE_CHECKLIST.md) for the tools repo
 
 ## v1.8
 

CHANGELOG.md

@@ -1,11 +1,14 @@
 # ![nf-core/tools](docs/images/nfcore-tools_logo.png) <!-- omit in toc -->
 
-[![GitHub Actions CI Status](https://github.com/nf-core/tools/workflows/CI%20tests/badge.svg)](https://github.com/nf-core/tools/actions)
+[![Python tests](https://github.com/nf-core/tools/workflows/Python%20tests/badge.svg?branch=master&event=push)](https://github.com/nf-core/tools/actions?query=workflow%3A%22Python+tests%22+branch%3Amaster)
 [![codecov](https://codecov.io/gh/nf-core/tools/branch/master/graph/badge.svg)](https://codecov.io/gh/nf-core/tools)
-[![install with bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg?style=flat-square)](http://bioconda.github.io/recipes/nf-core/README.html)
+[![install with Bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg)](http://bioconda.github.io/recipes/nf-core/README.html)
+[![install with PyPI](https://img.shields.io/badge/install%20with-PyPI-blue.svg)](https://pypi.org/project/nf-core/)
 
 A python package with helper tools for the nf-core community.
 
+> **Read this documentation on the nf-core website: [https://nf-co.re/tools](https://nf-co.re/tools)**
+
 ## Table of contents <!-- omit in toc -->
 
 * [`nf-core` tools installation](#installation)
@@ -243,7 +246,7 @@ INFO: Downloading centralised configs from GitHub
 
 INFO: Downloading 1 singularity container
 
-INFO: Building singularity image from dockerhub: docker://nfcore/methylseq:1.4
+INFO: Building singularity image from Docker Hub: docker://nfcore/methylseq:1.4
 INFO:    Converting OCI blobs to SIF format
 INFO:    Starting build...
 Getting image source signatures
@@ -534,8 +537,7 @@ with `--username` and `--repository`.
 
 To create the pull request, a personal access token is required for API authentication.
 These can be created at [https://github.com/settings/tokens](https://github.com/settings/tokens).
-Supply this using the `--auth-token` flag, or setting it as the environment variable `NF_CORE_BOT`:
-`export NF_CORE_BOT=my_auth_token`.
+Supply this using the `--auth-token` flag.
 
 Finally, if `--all` is supplied, then the command attempts to pull and synchronise all nf-core workflows.
 This is used by the nf-core/tools release automation to synchronise all nf-core pipelines
@@ -567,5 +569,5 @@ If you use `nf-core tools` in your work, please cite the `nf-core` publication a
 >
 > Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.
 >
-> _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x).  
+> _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x).
 > ReadCube: [Full Access Link](https://rdcu.be/b1GjZ)