-
Notifications
You must be signed in to change notification settings - Fork 186
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add configurable protocol support to ensembl reference download #2649
base: master
Are you sure you want to change the base?
feat: Add configurable protocol support to ensembl reference download #2649
Conversation
…embl reference data download and vep cache download.
Thanks for your contribution. |
I've run snakefmt, black and the linter over these changes. I believe this to be ready to merge. |
@pettyalex can you fix the fail test? |
…n. Add each new test case to test.py.
@fgvieira I fixed all the tests, and now see that what I had previously done introduced multiple ways to generate the desired output files. I also unintentionally ran black over |
def test_vep_cache_https_protocol(): | ||
run( | ||
"bio/vep/cache", | ||
["snakemake", "--cores", "1", "resources/vep/cache", "--use-conda", "-F"], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
["snakemake", "--cores", "1", "resources/vep/cache", "--use-conda", "-F"], | |
["snakemake", "--cores", "1", "resources/vep/cache", "--use-conda", "-F", "-s", "vep_cache_https.smk"], |
"-s", | ||
"vep_cache_https.smk", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"-s", | |
"vep_cache_https.smk", |
@@ -5428,17 +5461,26 @@ def test_ensembl_variation_with_contig_lengths(): | |||
|
|||
|
|||
@skip_if_not_modified | |||
def test_ega_fetch(): | |||
def test_ensembl_variation_old_release(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
def test_ensembl_variation_old_release(): | |
def test_ensembl_variation_old_release_https_protocol(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fantastic, thanks! Can you please also add the respective protocol lines to each Snakefile in the test folder? Those are used to generate the wrapper docs, from which people can copy paste into their workflows.
<!-- Ensure that the PR title follows conventional commit style (<type>: <description>)--> <!-- Possible types are here: https://github.com/commitizen/conventional-commit-types/blob/master/index.json --> <!-- Add a description of your PR here--> Allow for custom URLs (fix issues #366 and #2649). ### QC <!-- Make sure that you can tick the boxes below. --> * [x] I confirm that: For all wrappers added by this PR, * there is a test case which covers any introduced changes, * `input:` and `output:` file paths in the resulting rule can be changed arbitrarily, * either the wrapper can only use a single core, or the example rule contains a `threads: x` statement with `x` being a reasonable default, * rule names in the test case are in [snake_case](https://en.wikipedia.org/wiki/Snake_case) and somehow tell what the rule is about or match the tools purpose or name (e.g., `map_reads` for a step that maps reads), * all `environment.yaml` specifications follow [the respective best practices](https://stackoverflow.com/a/64594513/2352071), * the `environment.yaml` pinning has been updated by running `snakedeploy pin-conda-envs environment.yaml` on a linux machine, * wherever possible, command line arguments are inferred and set automatically (e.g. based on file extensions in `input:` or `output:`), * all fields of the example rules in the `Snakefile`s and their entries are explained via comments (`input:`/`output:`/`params:` etc.), * `stderr` and/or `stdout` are logged correctly (`log:`), depending on the wrapped tool, * temporary files are either written to a unique hidden folder in the working directory, or (better) stored where the Python function `tempfile.gettempdir()` points to (see [here](https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir); this also means that using any Python `tempfile` default behavior works), * the `meta.yaml` contains a link to the documentation of the respective tool or command, * `Snakefile`s pass the linting (`snakemake --lint`), * `Snakefile`s are formatted with [snakefmt](https://github.com/snakemake/snakefmt), * Python wrapper scripts are formatted with [black](https://black.readthedocs.io). * Conda environments use a minimal amount of channels, in recommended ordering. E.g. for bioconda, use (conda-forge, bioconda, nodefaults, as conda-forge should have highest priority and defaults channels are usually not needed because most packages are in conda-forge nowadays).
PR #2928 allowed for custom URLs when downloading from VEP cache. Maybe something similar can be done here for |
🤖 I have created a release \*beep\* \*boop\* --- ## [3.11.0](https://www.github.com/snakemake/snakemake-wrappers/compare/v3.10.2...v3.11.0) (2024-05-31) ### Features * add samtools markdup wrapper ([#2926](https://www.github.com/snakemake/snakemake-wrappers/issues/2926)) ([3c86bae](https://www.github.com/snakemake/snakemake-wrappers/commit/3c86baef4586b596c6c3375336828257e8bb815f)) * Add support for regions file and arbitrary FAI/GZI paths ([#2936](https://www.github.com/snakemake/snakemake-wrappers/issues/2936)) ([445b35f](https://www.github.com/snakemake/snakemake-wrappers/commit/445b35f314f035d8248ca84cb16b273b0e65847c)) * add wrapper to samtools collate ([#2929](https://www.github.com/snakemake/snakemake-wrappers/issues/2929)) ([0c7ae27](https://www.github.com/snakemake/snakemake-wrappers/commit/0c7ae27622e45167b9cc1aaf1be238b0d25429b9)) * All ngsderive subcommands ([#2732](https://www.github.com/snakemake/snakemake-wrappers/issues/2732)) ([847ab1d](https://www.github.com/snakemake/snakemake-wrappers/commit/847ab1d918c09908bcdb40fa3413b6dbc88d4803)) * auto infer run mode ([#2937](https://www.github.com/snakemake/snakemake-wrappers/issues/2937)) ([08bd3cd](https://www.github.com/snakemake/snakemake-wrappers/commit/08bd3cd79243c4c0636f88d229667e58595611bd)) * Goleft indexcov ([#2734](https://www.github.com/snakemake/snakemake-wrappers/issues/2734)) ([ebef6f8](https://www.github.com/snakemake/snakemake-wrappers/commit/ebef6f8259e89d9a06b2264a5d0a34da23d2e84e)) ### Bug Fixes * bamcoverage without effective genome size ([#2941](https://www.github.com/snakemake/snakemake-wrappers/issues/2941)) ([b6ad55f](https://www.github.com/snakemake/snakemake-wrappers/commit/b6ad55f4d2c702ecc904d6b40b4b35b2463c895a)) * issue [#366](https://www.github.com/snakemake/snakemake-wrappers/issues/366) and [#2649](https://www.github.com/snakemake/snakemake-wrappers/issues/2649) ([#2928](https://www.github.com/snakemake/snakemake-wrappers/issues/2928)) ([e10ab57](https://www.github.com/snakemake/snakemake-wrappers/commit/e10ab571a63b36ce1295e50b6f6b7e4742e0beae)) * remove NA string replacement, fixed upstream ([#2940](https://www.github.com/snakemake/snakemake-wrappers/issues/2940)) ([8f4d223](https://www.github.com/snakemake/snakemake-wrappers/commit/8f4d223ec6c1e1e2715708d0532399ede4ed2727)) ### Performance Improvements * autobump bio/bismark/bam2nuc ([#2943](https://www.github.com/snakemake/snakemake-wrappers/issues/2943)) ([9228479](https://www.github.com/snakemake/snakemake-wrappers/commit/9228479896c2410ae6a58849bf59e5f01bc5e712)) * autobump bio/bismark/bismark ([#2948](https://www.github.com/snakemake/snakemake-wrappers/issues/2948)) ([d9abb20](https://www.github.com/snakemake/snakemake-wrappers/commit/d9abb20e0e002d9ec16507d86211b0555a2546ba)) * autobump bio/bismark/bismark_genome_preparation ([#2945](https://www.github.com/snakemake/snakemake-wrappers/issues/2945)) ([899ae0d](https://www.github.com/snakemake/snakemake-wrappers/commit/899ae0d96d04fa56a7ba5989d9dea18be0d3ebc6)) * autobump bio/bismark/bismark_methylation_extractor ([#2944](https://www.github.com/snakemake/snakemake-wrappers/issues/2944)) ([1b232d2](https://www.github.com/snakemake/snakemake-wrappers/commit/1b232d24816a40f75568bec224eff4e164b3f953)) * autobump bio/bismark/bismark2bedGraph ([#2947](https://www.github.com/snakemake/snakemake-wrappers/issues/2947)) ([ab29098](https://www.github.com/snakemake/snakemake-wrappers/commit/ab29098757c3eef17884f198234ec456e19414f9)) * autobump bio/bismark/bismark2report ([#2951](https://www.github.com/snakemake/snakemake-wrappers/issues/2951)) ([13dd155](https://www.github.com/snakemake/snakemake-wrappers/commit/13dd155a20025b8aaaa80ce3747e6822562bb6b1)) * autobump bio/bismark/bismark2summary ([#2946](https://www.github.com/snakemake/snakemake-wrappers/issues/2946)) ([bfb3d30](https://www.github.com/snakemake/snakemake-wrappers/commit/bfb3d305daa519faee4ad4ea7dbd313941f4c128)) * autobump bio/bismark/deduplicate_bismark ([#2950](https://www.github.com/snakemake/snakemake-wrappers/issues/2950)) ([bcf3c06](https://www.github.com/snakemake/snakemake-wrappers/commit/bcf3c06887443dae9749254f0e8d34e30f880b7e)) * autobump bio/bowtie2/align ([#2949](https://www.github.com/snakemake/snakemake-wrappers/issues/2949)) ([6b0593f](https://www.github.com/snakemake/snakemake-wrappers/commit/6b0593facfcae0b9c92cb1f31423426752c0230d)) * autobump bio/bowtie2/build ([#2942](https://www.github.com/snakemake/snakemake-wrappers/issues/2942)) ([5b3ca20](https://www.github.com/snakemake/snakemake-wrappers/commit/5b3ca20635bd0e427a548e68cd8a40fdb0047f10)) * autobump bio/bwa-mem2/mem-samblaster ([#2962](https://www.github.com/snakemake/snakemake-wrappers/issues/2962)) ([ff613f3](https://www.github.com/snakemake/snakemake-wrappers/commit/ff613f37f857f23d19e4dc0cace22b02113a7e07)) * autobump bio/bwa/mem-samblaster ([#2961](https://www.github.com/snakemake/snakemake-wrappers/issues/2961)) ([83754dd](https://www.github.com/snakemake/snakemake-wrappers/commit/83754dd9f158ea978faa1ed33668a6dda648cdb7)) * autobump bio/fastq_screen ([#2952](https://www.github.com/snakemake/snakemake-wrappers/issues/2952)) ([e45fcac](https://www.github.com/snakemake/snakemake-wrappers/commit/e45fcac221ef45f345a516bff905a53c864e7355)) * autobump bio/freebayes ([#2963](https://www.github.com/snakemake/snakemake-wrappers/issues/2963)) ([7a0d553](https://www.github.com/snakemake/snakemake-wrappers/commit/7a0d553e09921cebc77b54d314c211b91099f7bd)) * autobump bio/gatk/applybqsr ([#2938](https://www.github.com/snakemake/snakemake-wrappers/issues/2938)) ([498c67f](https://www.github.com/snakemake/snakemake-wrappers/commit/498c67fb12691b0021a7b234dfc279957b46708c)) * autobump bio/gatk3/realignertargetcreator ([#2931](https://www.github.com/snakemake/snakemake-wrappers/issues/2931)) ([2d2835c](https://www.github.com/snakemake/snakemake-wrappers/commit/2d2835c51f974f7f45420f832b8084520d2d3ec0)) * autobump bio/gdc-api/bam-slicing ([#2954](https://www.github.com/snakemake/snakemake-wrappers/issues/2954)) ([32d25c1](https://www.github.com/snakemake/snakemake-wrappers/commit/32d25c1560b4c581694c49dc6d3a596e5ca581ab)) * autobump bio/goleft/indexcov ([#2930](https://www.github.com/snakemake/snakemake-wrappers/issues/2930)) ([5ccbdb0](https://www.github.com/snakemake/snakemake-wrappers/commit/5ccbdb078a835ad0c2fd3e9e63677103fe216eeb)) * autobump bio/gseapy/gsea ([#2953](https://www.github.com/snakemake/snakemake-wrappers/issues/2953)) ([dda3d16](https://www.github.com/snakemake/snakemake-wrappers/commit/dda3d16c9b49145bd58d287db1021c41d5a24710)) * autobump bio/hifiasm ([#2932](https://www.github.com/snakemake/snakemake-wrappers/issues/2932)) ([3722de7](https://www.github.com/snakemake/snakemake-wrappers/commit/3722de7000d0b74a3c56bfdb27aad6cf318562ee)) * autobump bio/multiqc ([#2955](https://www.github.com/snakemake/snakemake-wrappers/issues/2955)) ([91ad107](https://www.github.com/snakemake/snakemake-wrappers/commit/91ad1079fa03b07f8ebb9ebf21fe92d21c8a3c13)) * autobump bio/ngsderive ([#2964](https://www.github.com/snakemake/snakemake-wrappers/issues/2964)) ([df82b38](https://www.github.com/snakemake/snakemake-wrappers/commit/df82b381a49b7810087f73b7c83db4fff6d57b66)) * autobump bio/open-cravat/module ([#2966](https://www.github.com/snakemake/snakemake-wrappers/issues/2966)) ([f1ffa3e](https://www.github.com/snakemake/snakemake-wrappers/commit/f1ffa3eee366cba828314cc9a368ba69f17d98c2)) * autobump bio/open-cravat/run ([#2965](https://www.github.com/snakemake/snakemake-wrappers/issues/2965)) ([9b371ca](https://www.github.com/snakemake/snakemake-wrappers/commit/9b371ca6014a38cf23e78598638e4c709a5e82af)) * autobump bio/ptrimmer ([#2933](https://www.github.com/snakemake/snakemake-wrappers/issues/2933)) ([cd37e0b](https://www.github.com/snakemake/snakemake-wrappers/commit/cd37e0bb19e341ad40560828f3c052ef69529088)) * autobump bio/rasusa ([#2967](https://www.github.com/snakemake/snakemake-wrappers/issues/2967)) ([9f25643](https://www.github.com/snakemake/snakemake-wrappers/commit/9f2564323429bea84dfb0b01a5b9f53b37fa397c)) * autobump bio/rbt/csvreport ([#2956](https://www.github.com/snakemake/snakemake-wrappers/issues/2956)) ([b2a469d](https://www.github.com/snakemake/snakemake-wrappers/commit/b2a469dc907c8f3761399affd29ca53f012cc3ee)) * autobump bio/sambamba/flagstat ([#2973](https://www.github.com/snakemake/snakemake-wrappers/issues/2973)) ([4575792](https://www.github.com/snakemake/snakemake-wrappers/commit/4575792deb144dd80401d46bc2fb72175976305a)) * autobump bio/sambamba/index ([#2971](https://www.github.com/snakemake/snakemake-wrappers/issues/2971)) ([18359ac](https://www.github.com/snakemake/snakemake-wrappers/commit/18359ac437bb9f1065c703b3bf2da1d3da9548be)) * autobump bio/sambamba/markdup ([#2968](https://www.github.com/snakemake/snakemake-wrappers/issues/2968)) ([191542d](https://www.github.com/snakemake/snakemake-wrappers/commit/191542d475853f6f19495e136e105970928a9fd8)) * autobump bio/sambamba/merge ([#2975](https://www.github.com/snakemake/snakemake-wrappers/issues/2975)) ([adf9a28](https://www.github.com/snakemake/snakemake-wrappers/commit/adf9a28b64f87aec3fa8286f356f44b6bb77561e)) * autobump bio/sambamba/slice ([#2972](https://www.github.com/snakemake/snakemake-wrappers/issues/2972)) ([88ac3f6](https://www.github.com/snakemake/snakemake-wrappers/commit/88ac3f6dfae93606f74ae8c88b6f8b9e2d059a02)) * autobump bio/sambamba/sort ([#2976](https://www.github.com/snakemake/snakemake-wrappers/issues/2976)) ([990e841](https://www.github.com/snakemake/snakemake-wrappers/commit/990e8416b730267a65ba5a97b71537655bbfe6db)) * autobump bio/sambamba/view ([#2970](https://www.github.com/snakemake/snakemake-wrappers/issues/2970)) ([613e55d](https://www.github.com/snakemake/snakemake-wrappers/commit/613e55d915215348651e878fe37f6533e6d77363)) * autobump bio/seqkit ([#2969](https://www.github.com/snakemake/snakemake-wrappers/issues/2969)) ([e25b840](https://www.github.com/snakemake/snakemake-wrappers/commit/e25b8409af3cc9e591b5106f1ebb00289f628207)) * autobump bio/sra-tools/fasterq-dump ([#2974](https://www.github.com/snakemake/snakemake-wrappers/issues/2974)) ([d8b4b5f](https://www.github.com/snakemake/snakemake-wrappers/commit/d8b4b5fab114c66bf712ee271b00f831a5ed558d)) * autobump bio/unicycler ([#2957](https://www.github.com/snakemake/snakemake-wrappers/issues/2957)) ([185ce7e](https://www.github.com/snakemake/snakemake-wrappers/commit/185ce7e4bd6adbdf840effa665f62e358865c8b8)) * autobump bio/vep/annotate ([#2958](https://www.github.com/snakemake/snakemake-wrappers/issues/2958)) ([728658d](https://www.github.com/snakemake/snakemake-wrappers/commit/728658d5fae40ea784f7b8d3c567c4f7240ef7c7)) * autobump bio/vep/cache ([#2959](https://www.github.com/snakemake/snakemake-wrappers/issues/2959)) ([f4e5b66](https://www.github.com/snakemake/snakemake-wrappers/commit/f4e5b66f8765d09d51d4a640d2d775d781c3018f)) * autobump bio/whatshap/haplotag ([#2934](https://www.github.com/snakemake/snakemake-wrappers/issues/2934)) ([f0b638a](https://www.github.com/snakemake/snakemake-wrappers/commit/f0b638a55280c0727ccef5aab24e03b3d565f290)) * Update Datavzrd to 2.36.12 ([#2924](https://www.github.com/snakemake/snakemake-wrappers/issues/2924)) ([beb9d22](https://www.github.com/snakemake/snakemake-wrappers/commit/beb9d2231c5d59ba74f23f56bbfc5e004aa72331)) * Use samtools collate in fastq separate wrapper ([#2960](https://www.github.com/snakemake/snakemake-wrappers/issues/2960)) ([9c8cf81](https://www.github.com/snakemake/snakemake-wrappers/commit/9c8cf81c1894fc019ffbf0f906eb88e0960c3e7d)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
<!-- Ensure that the PR title follows conventional commit style (<type>: <description>)--> <!-- Possible types are here: https://github.com/commitizen/conventional-commit-types/blob/master/index.json --> <!-- Add a description of your PR here--> Allow for custom URLs (fix issues snakemake#366 and snakemake#2649). ### QC <!-- Make sure that you can tick the boxes below. --> * [x] I confirm that: For all wrappers added by this PR, * there is a test case which covers any introduced changes, * `input:` and `output:` file paths in the resulting rule can be changed arbitrarily, * either the wrapper can only use a single core, or the example rule contains a `threads: x` statement with `x` being a reasonable default, * rule names in the test case are in [snake_case](https://en.wikipedia.org/wiki/Snake_case) and somehow tell what the rule is about or match the tools purpose or name (e.g., `map_reads` for a step that maps reads), * all `environment.yaml` specifications follow [the respective best practices](https://stackoverflow.com/a/64594513/2352071), * the `environment.yaml` pinning has been updated by running `snakedeploy pin-conda-envs environment.yaml` on a linux machine, * wherever possible, command line arguments are inferred and set automatically (e.g. based on file extensions in `input:` or `output:`), * all fields of the example rules in the `Snakefile`s and their entries are explained via comments (`input:`/`output:`/`params:` etc.), * `stderr` and/or `stdout` are logged correctly (`log:`), depending on the wrapped tool, * temporary files are either written to a unique hidden folder in the working directory, or (better) stored where the Python function `tempfile.gettempdir()` points to (see [here](https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir); this also means that using any Python `tempfile` default behavior works), * the `meta.yaml` contains a link to the documentation of the respective tool or command, * `Snakefile`s pass the linting (`snakemake --lint`), * `Snakefile`s are formatted with [snakefmt](https://github.com/snakemake/snakefmt), * Python wrapper scripts are formatted with [black](https://black.readthedocs.io). * Conda environments use a minimal amount of channels, in recommended ordering. E.g. for bioconda, use (conda-forge, bioconda, nodefaults, as conda-forge should have highest priority and defaults channels are usually not needed because most packages are in conda-forge nowadays).
🤖 I have created a release \*beep\* \*boop\* --- ## [3.11.0](https://www.github.com/snakemake/snakemake-wrappers/compare/v3.10.2...v3.11.0) (2024-05-31) ### Features * add samtools markdup wrapper ([snakemake#2926](https://www.github.com/snakemake/snakemake-wrappers/issues/2926)) ([3c86bae](https://www.github.com/snakemake/snakemake-wrappers/commit/3c86baef4586b596c6c3375336828257e8bb815f)) * Add support for regions file and arbitrary FAI/GZI paths ([snakemake#2936](https://www.github.com/snakemake/snakemake-wrappers/issues/2936)) ([445b35f](https://www.github.com/snakemake/snakemake-wrappers/commit/445b35f314f035d8248ca84cb16b273b0e65847c)) * add wrapper to samtools collate ([snakemake#2929](https://www.github.com/snakemake/snakemake-wrappers/issues/2929)) ([0c7ae27](https://www.github.com/snakemake/snakemake-wrappers/commit/0c7ae27622e45167b9cc1aaf1be238b0d25429b9)) * All ngsderive subcommands ([snakemake#2732](https://www.github.com/snakemake/snakemake-wrappers/issues/2732)) ([847ab1d](https://www.github.com/snakemake/snakemake-wrappers/commit/847ab1d918c09908bcdb40fa3413b6dbc88d4803)) * auto infer run mode ([snakemake#2937](https://www.github.com/snakemake/snakemake-wrappers/issues/2937)) ([08bd3cd](https://www.github.com/snakemake/snakemake-wrappers/commit/08bd3cd79243c4c0636f88d229667e58595611bd)) * Goleft indexcov ([snakemake#2734](https://www.github.com/snakemake/snakemake-wrappers/issues/2734)) ([ebef6f8](https://www.github.com/snakemake/snakemake-wrappers/commit/ebef6f8259e89d9a06b2264a5d0a34da23d2e84e)) ### Bug Fixes * bamcoverage without effective genome size ([snakemake#2941](https://www.github.com/snakemake/snakemake-wrappers/issues/2941)) ([b6ad55f](https://www.github.com/snakemake/snakemake-wrappers/commit/b6ad55f4d2c702ecc904d6b40b4b35b2463c895a)) * issue [snakemake#366](https://www.github.com/snakemake/snakemake-wrappers/issues/366) and [snakemake#2649](https://www.github.com/snakemake/snakemake-wrappers/issues/2649) ([snakemake#2928](https://www.github.com/snakemake/snakemake-wrappers/issues/2928)) ([e10ab57](https://www.github.com/snakemake/snakemake-wrappers/commit/e10ab571a63b36ce1295e50b6f6b7e4742e0beae)) * remove NA string replacement, fixed upstream ([snakemake#2940](https://www.github.com/snakemake/snakemake-wrappers/issues/2940)) ([8f4d223](https://www.github.com/snakemake/snakemake-wrappers/commit/8f4d223ec6c1e1e2715708d0532399ede4ed2727)) ### Performance Improvements * autobump bio/bismark/bam2nuc ([snakemake#2943](https://www.github.com/snakemake/snakemake-wrappers/issues/2943)) ([9228479](https://www.github.com/snakemake/snakemake-wrappers/commit/9228479896c2410ae6a58849bf59e5f01bc5e712)) * autobump bio/bismark/bismark ([snakemake#2948](https://www.github.com/snakemake/snakemake-wrappers/issues/2948)) ([d9abb20](https://www.github.com/snakemake/snakemake-wrappers/commit/d9abb20e0e002d9ec16507d86211b0555a2546ba)) * autobump bio/bismark/bismark_genome_preparation ([snakemake#2945](https://www.github.com/snakemake/snakemake-wrappers/issues/2945)) ([899ae0d](https://www.github.com/snakemake/snakemake-wrappers/commit/899ae0d96d04fa56a7ba5989d9dea18be0d3ebc6)) * autobump bio/bismark/bismark_methylation_extractor ([snakemake#2944](https://www.github.com/snakemake/snakemake-wrappers/issues/2944)) ([1b232d2](https://www.github.com/snakemake/snakemake-wrappers/commit/1b232d24816a40f75568bec224eff4e164b3f953)) * autobump bio/bismark/bismark2bedGraph ([snakemake#2947](https://www.github.com/snakemake/snakemake-wrappers/issues/2947)) ([ab29098](https://www.github.com/snakemake/snakemake-wrappers/commit/ab29098757c3eef17884f198234ec456e19414f9)) * autobump bio/bismark/bismark2report ([snakemake#2951](https://www.github.com/snakemake/snakemake-wrappers/issues/2951)) ([13dd155](https://www.github.com/snakemake/snakemake-wrappers/commit/13dd155a20025b8aaaa80ce3747e6822562bb6b1)) * autobump bio/bismark/bismark2summary ([snakemake#2946](https://www.github.com/snakemake/snakemake-wrappers/issues/2946)) ([bfb3d30](https://www.github.com/snakemake/snakemake-wrappers/commit/bfb3d305daa519faee4ad4ea7dbd313941f4c128)) * autobump bio/bismark/deduplicate_bismark ([snakemake#2950](https://www.github.com/snakemake/snakemake-wrappers/issues/2950)) ([bcf3c06](https://www.github.com/snakemake/snakemake-wrappers/commit/bcf3c06887443dae9749254f0e8d34e30f880b7e)) * autobump bio/bowtie2/align ([snakemake#2949](https://www.github.com/snakemake/snakemake-wrappers/issues/2949)) ([6b0593f](https://www.github.com/snakemake/snakemake-wrappers/commit/6b0593facfcae0b9c92cb1f31423426752c0230d)) * autobump bio/bowtie2/build ([snakemake#2942](https://www.github.com/snakemake/snakemake-wrappers/issues/2942)) ([5b3ca20](https://www.github.com/snakemake/snakemake-wrappers/commit/5b3ca20635bd0e427a548e68cd8a40fdb0047f10)) * autobump bio/bwa-mem2/mem-samblaster ([snakemake#2962](https://www.github.com/snakemake/snakemake-wrappers/issues/2962)) ([ff613f3](https://www.github.com/snakemake/snakemake-wrappers/commit/ff613f37f857f23d19e4dc0cace22b02113a7e07)) * autobump bio/bwa/mem-samblaster ([snakemake#2961](https://www.github.com/snakemake/snakemake-wrappers/issues/2961)) ([83754dd](https://www.github.com/snakemake/snakemake-wrappers/commit/83754dd9f158ea978faa1ed33668a6dda648cdb7)) * autobump bio/fastq_screen ([snakemake#2952](https://www.github.com/snakemake/snakemake-wrappers/issues/2952)) ([e45fcac](https://www.github.com/snakemake/snakemake-wrappers/commit/e45fcac221ef45f345a516bff905a53c864e7355)) * autobump bio/freebayes ([snakemake#2963](https://www.github.com/snakemake/snakemake-wrappers/issues/2963)) ([7a0d553](https://www.github.com/snakemake/snakemake-wrappers/commit/7a0d553e09921cebc77b54d314c211b91099f7bd)) * autobump bio/gatk/applybqsr ([snakemake#2938](https://www.github.com/snakemake/snakemake-wrappers/issues/2938)) ([498c67f](https://www.github.com/snakemake/snakemake-wrappers/commit/498c67fb12691b0021a7b234dfc279957b46708c)) * autobump bio/gatk3/realignertargetcreator ([snakemake#2931](https://www.github.com/snakemake/snakemake-wrappers/issues/2931)) ([2d2835c](https://www.github.com/snakemake/snakemake-wrappers/commit/2d2835c51f974f7f45420f832b8084520d2d3ec0)) * autobump bio/gdc-api/bam-slicing ([snakemake#2954](https://www.github.com/snakemake/snakemake-wrappers/issues/2954)) ([32d25c1](https://www.github.com/snakemake/snakemake-wrappers/commit/32d25c1560b4c581694c49dc6d3a596e5ca581ab)) * autobump bio/goleft/indexcov ([snakemake#2930](https://www.github.com/snakemake/snakemake-wrappers/issues/2930)) ([5ccbdb0](https://www.github.com/snakemake/snakemake-wrappers/commit/5ccbdb078a835ad0c2fd3e9e63677103fe216eeb)) * autobump bio/gseapy/gsea ([snakemake#2953](https://www.github.com/snakemake/snakemake-wrappers/issues/2953)) ([dda3d16](https://www.github.com/snakemake/snakemake-wrappers/commit/dda3d16c9b49145bd58d287db1021c41d5a24710)) * autobump bio/hifiasm ([snakemake#2932](https://www.github.com/snakemake/snakemake-wrappers/issues/2932)) ([3722de7](https://www.github.com/snakemake/snakemake-wrappers/commit/3722de7000d0b74a3c56bfdb27aad6cf318562ee)) * autobump bio/multiqc ([snakemake#2955](https://www.github.com/snakemake/snakemake-wrappers/issues/2955)) ([91ad107](https://www.github.com/snakemake/snakemake-wrappers/commit/91ad1079fa03b07f8ebb9ebf21fe92d21c8a3c13)) * autobump bio/ngsderive ([snakemake#2964](https://www.github.com/snakemake/snakemake-wrappers/issues/2964)) ([df82b38](https://www.github.com/snakemake/snakemake-wrappers/commit/df82b381a49b7810087f73b7c83db4fff6d57b66)) * autobump bio/open-cravat/module ([snakemake#2966](https://www.github.com/snakemake/snakemake-wrappers/issues/2966)) ([f1ffa3e](https://www.github.com/snakemake/snakemake-wrappers/commit/f1ffa3eee366cba828314cc9a368ba69f17d98c2)) * autobump bio/open-cravat/run ([snakemake#2965](https://www.github.com/snakemake/snakemake-wrappers/issues/2965)) ([9b371ca](https://www.github.com/snakemake/snakemake-wrappers/commit/9b371ca6014a38cf23e78598638e4c709a5e82af)) * autobump bio/ptrimmer ([snakemake#2933](https://www.github.com/snakemake/snakemake-wrappers/issues/2933)) ([cd37e0b](https://www.github.com/snakemake/snakemake-wrappers/commit/cd37e0bb19e341ad40560828f3c052ef69529088)) * autobump bio/rasusa ([snakemake#2967](https://www.github.com/snakemake/snakemake-wrappers/issues/2967)) ([9f25643](https://www.github.com/snakemake/snakemake-wrappers/commit/9f2564323429bea84dfb0b01a5b9f53b37fa397c)) * autobump bio/rbt/csvreport ([snakemake#2956](https://www.github.com/snakemake/snakemake-wrappers/issues/2956)) ([b2a469d](https://www.github.com/snakemake/snakemake-wrappers/commit/b2a469dc907c8f3761399affd29ca53f012cc3ee)) * autobump bio/sambamba/flagstat ([snakemake#2973](https://www.github.com/snakemake/snakemake-wrappers/issues/2973)) ([4575792](https://www.github.com/snakemake/snakemake-wrappers/commit/4575792deb144dd80401d46bc2fb72175976305a)) * autobump bio/sambamba/index ([snakemake#2971](https://www.github.com/snakemake/snakemake-wrappers/issues/2971)) ([18359ac](https://www.github.com/snakemake/snakemake-wrappers/commit/18359ac437bb9f1065c703b3bf2da1d3da9548be)) * autobump bio/sambamba/markdup ([snakemake#2968](https://www.github.com/snakemake/snakemake-wrappers/issues/2968)) ([191542d](https://www.github.com/snakemake/snakemake-wrappers/commit/191542d475853f6f19495e136e105970928a9fd8)) * autobump bio/sambamba/merge ([snakemake#2975](https://www.github.com/snakemake/snakemake-wrappers/issues/2975)) ([adf9a28](https://www.github.com/snakemake/snakemake-wrappers/commit/adf9a28b64f87aec3fa8286f356f44b6bb77561e)) * autobump bio/sambamba/slice ([snakemake#2972](https://www.github.com/snakemake/snakemake-wrappers/issues/2972)) ([88ac3f6](https://www.github.com/snakemake/snakemake-wrappers/commit/88ac3f6dfae93606f74ae8c88b6f8b9e2d059a02)) * autobump bio/sambamba/sort ([snakemake#2976](https://www.github.com/snakemake/snakemake-wrappers/issues/2976)) ([990e841](https://www.github.com/snakemake/snakemake-wrappers/commit/990e8416b730267a65ba5a97b71537655bbfe6db)) * autobump bio/sambamba/view ([snakemake#2970](https://www.github.com/snakemake/snakemake-wrappers/issues/2970)) ([613e55d](https://www.github.com/snakemake/snakemake-wrappers/commit/613e55d915215348651e878fe37f6533e6d77363)) * autobump bio/seqkit ([snakemake#2969](https://www.github.com/snakemake/snakemake-wrappers/issues/2969)) ([e25b840](https://www.github.com/snakemake/snakemake-wrappers/commit/e25b8409af3cc9e591b5106f1ebb00289f628207)) * autobump bio/sra-tools/fasterq-dump ([snakemake#2974](https://www.github.com/snakemake/snakemake-wrappers/issues/2974)) ([d8b4b5f](https://www.github.com/snakemake/snakemake-wrappers/commit/d8b4b5fab114c66bf712ee271b00f831a5ed558d)) * autobump bio/unicycler ([snakemake#2957](https://www.github.com/snakemake/snakemake-wrappers/issues/2957)) ([185ce7e](https://www.github.com/snakemake/snakemake-wrappers/commit/185ce7e4bd6adbdf840effa665f62e358865c8b8)) * autobump bio/vep/annotate ([snakemake#2958](https://www.github.com/snakemake/snakemake-wrappers/issues/2958)) ([728658d](https://www.github.com/snakemake/snakemake-wrappers/commit/728658d5fae40ea784f7b8d3c567c4f7240ef7c7)) * autobump bio/vep/cache ([snakemake#2959](https://www.github.com/snakemake/snakemake-wrappers/issues/2959)) ([f4e5b66](https://www.github.com/snakemake/snakemake-wrappers/commit/f4e5b66f8765d09d51d4a640d2d775d781c3018f)) * autobump bio/whatshap/haplotag ([snakemake#2934](https://www.github.com/snakemake/snakemake-wrappers/issues/2934)) ([f0b638a](https://www.github.com/snakemake/snakemake-wrappers/commit/f0b638a55280c0727ccef5aab24e03b3d565f290)) * Update Datavzrd to 2.36.12 ([snakemake#2924](https://www.github.com/snakemake/snakemake-wrappers/issues/2924)) ([beb9d22](https://www.github.com/snakemake/snakemake-wrappers/commit/beb9d2231c5d59ba74f23f56bbfc5e004aa72331)) * Use samtools collate in fastq separate wrapper ([snakemake#2960](https://www.github.com/snakemake/snakemake-wrappers/issues/2960)) ([9c8cf81](https://www.github.com/snakemake/snakemake-wrappers/commit/9c8cf81c1894fc019ffbf0f906eb88e0960c3e7d)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Can you fix the conflicts? |
This PR adds support for configurable protocol, ftp, http, or https to ensembl reference data download and vep cache download.
I was evaluating snakemake-wrapper and widely used workflows as a tool for my group to potentially use, and immediately ran into our firewall rules. By default we're able to make outgoing HTTP and HTTPS requests, but not FTP. I can request a change to our firewall, but it also would be helpful if these rules that download reference files were able to download them over HTTP, as ftp.ensembl.org has has been available over HTTP for a very long time.
If you'd prefer, I can create an issue to track this. I also intend to update test cases on these wrappers, and if you are happy with this pattern I could apply it across all wrappers in this repository that are currently hard-coded to make ftp requests.
QC
For all wrappers added by this PR,
input:
andoutput:
file paths in the resulting rule can be changed arbitrarily,threads: x
statement withx
being a reasonable default,map_reads
for a step that maps reads),environment.yaml
specifications follow the respective best practices,environment.yaml
pinning has been updated by runningsnakedeploy pin-conda-envs environment.yaml
on a linux machine,input:
oroutput:
),Snakefile
s and their entries are explained via comments (input:
/output:
/params:
etc.),stderr
and/orstdout
are logged correctly (log:
), depending on the wrapped tool,tempfile.gettempdir()
points to (see here; this also means that using any Pythontempfile
default behavior works),meta.yaml
contains a link to the documentation of the respective tool or command,Snakefile
s pass the linting (snakemake --lint
),Snakefile
s are formatted with snakefmt,