-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
5519727
commit 27995c9
Showing
46 changed files
with
1,486 additions
and
0 deletions.
There are no files selected for viewing
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
poetry.lock |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
[bumpversion] | ||
current_version = 1.1.0-dev1 | ||
commit = False | ||
tag = False | ||
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\-(?P<release>[a-z]+)(?P<dev>\d+))? | ||
serialize = | ||
{major}.{minor}.{patch}-{release}{dev} | ||
{major}.{minor}.{patch} | ||
|
||
[bumpversion:part:release] | ||
optional_value = _ | ||
first_value = dev | ||
values = | ||
dev | ||
_ | ||
|
||
[bumpversion:part:dev] | ||
|
||
[bumpversion:file:pyproject.toml] | ||
search = version = "{current_version}" | ||
replace = version = "{new_version}" | ||
|
||
[bumpversion:file:README.md] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# CHANGELOG | ||
|
||
# 1.0.0 | ||
|
||
* Generate plugins from templates using cookiecutter. | ||
|
||
# 1.1.0 | ||
|
||
* Generate plugins following updated [standard guidelines](https://labshare.atlassian.net/wiki/spaces/WIPP/pages/3275980801/Python+Plugin+Standards) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,111 @@ | ||
# WIPP Plugin Cookie Cutter (for Python) (1.1.0-dev1) | ||
|
||
This repository is a cookie cutter template that creates the basic scaffold structure of a | ||
polus plugin and add it to the polus plugins directory structure. | ||
|
||
## How to use | ||
1. Clone `tools-templates` and change to the polus-plugins directory | ||
2. `cd tabular-templates/python-template` | ||
3. (optional) Install poetry if not available. | ||
4. (optional) Create a dedicated environment with conda or venv. | ||
5. Install the dependencies: `poetry install` | ||
6. Modify `cookiecutter.json` to include author and plugin information.`plugin_package` should always start with `polus.tabular`. | ||
** NOTE: ** Do not edit values in brackets ({}) as they are edited by cookiecutter directly. | ||
Those are automatically generated from the previous entries. If your plugin is called | ||
"Awesome Function", then the plugin folder and docker container will have the name `awesome-function-plugin`. | ||
7. Create your plugin skeleton: `python -m cookiecutter . --no-input` | ||
|
||
|
||
## Plugin Standard | ||
The generated plugin will be compatible with polus most up-to-date guidelines : | ||
see [standard guidelines](https://labshare.atlassian.net/wiki/spaces/WIPP/pages/3275980801/Python+Plugin+Standards) | ||
|
||
The code generated provides out-of-box support for : | ||
- customizing the plugin code. | ||
- implementing tests. | ||
- creating and running a container. | ||
- managing versioning. | ||
- updating documentation (README, CHANGELOG). | ||
- maintaining a WIPP manifest (plugin.json). | ||
|
||
|
||
## Executing the plugin | ||
|
||
The plugin should be run as a package. | ||
To install the package : | ||
|
||
`pip install .` | ||
|
||
The skeleton code can be run this way : | ||
From the plugin's top directory (with the default values): | ||
|
||
`python -m polus.tabular.package1.package2.awesome_function -i /tmp/inp -o /tmp/out` | ||
|
||
This should print some logs with the provided inputs and outputs and return. | ||
|
||
## Running tests | ||
Plugin's developer should use `pytest`. | ||
Some simple tests have been added to the template as examples. | ||
Before submitting a PR to `tabular-tools`, other unit tests should be created and added to the `tests` | ||
directory. | ||
|
||
To run tests : | ||
|
||
From the plugin's top directory, type `python -m pytest`. | ||
Depending on how you have set up your environment, you may be able to run the pytest cli directly `pytest`. See pytest doc for how the project source directory is scanned to collect tests. | ||
This should run a test successfully and return. | ||
|
||
|
||
## Creating and running a container | ||
|
||
` ./build-docker.sh && ./run-plugin.sh` | ||
|
||
Build the docker image and run the container. | ||
|
||
### DockerFile | ||
A docker image is build from a base image with common dependencies pre-installed. | ||
The image entrypoint will run the plugin's package entrypoint. | ||
|
||
### build-docker.sh | ||
Run this script to build the container. | ||
|
||
### run-plugin.sh | ||
Run the container locally. | ||
|
||
|
||
## Customize the plugin | ||
|
||
### Project code | ||
|
||
A set of common dependencies are added to `pyproject.toml`. | ||
Update according to your needs. | ||
|
||
### Managing versioning | ||
|
||
Making sure that the file version is consistent across files in a plugin can be | ||
challenging, so the Python template now uses | ||
[bump2version](https://github.com/c4urself/bump2version) | ||
to help manage versioning. This automatically changes the `VERSION` and | ||
`plugin.json` files to the next version, preventing you from having to remember | ||
to change the version everywhere. The `bumpversion.cfg` can be modified to | ||
change the version in other files as well. | ||
|
||
To use this feature: | ||
`bump2version --config-file bumpversion.cfg` | ||
|
||
### Documentation | ||
|
||
#### README.md | ||
|
||
A basic description of what the plugin does. This should define all the inputs | ||
and outputs. | ||
|
||
#### CHANGELOG.md | ||
|
||
Documents updates made to the plugin. | ||
|
||
|
||
### WIPP manifest (plugin.json). | ||
|
||
This file defines the input and output variables for WIPP, and defines the UI | ||
components showed to the user. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
1.1.0-dev0 |
29 changes: 29 additions & 0 deletions
29
tabular-templates/python-template/awesome-function-tool/.bumpversion.cfg
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
[bumpversion] | ||
current_version = 0.1.0 | ||
commit = False | ||
tag = False | ||
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\-(?P<release>[a-z]+)(?P<dev>\d+))? | ||
serialize = | ||
{major}.{minor}.{patch}-{release}{dev} | ||
{major}.{minor}.{patch} | ||
|
||
[bumpversion:part:release] | ||
optional_value = _ | ||
first_value = dev | ||
values = | ||
dev | ||
_ | ||
|
||
[bumpversion:part:dev] | ||
|
||
[bumpversion:file:pyproject.toml] | ||
search = version = "{current_version}" | ||
replace = version = "{new_version}" | ||
|
||
[bumpversion:file:VERSION] | ||
|
||
[bumpversion:file:README.md] | ||
|
||
[bumpversion:file:plugin.json] | ||
|
||
[bumpversion:file:src/polus/tabular/package1/package2/awesome_function/__init__.py] |
4 changes: 4 additions & 0 deletions
4
tabular-templates/python-template/awesome-function-tool/.dockerignore
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
.venv | ||
out | ||
tests | ||
__pycache__ |
1 change: 1 addition & 0 deletions
1
tabular-templates/python-template/awesome-function-tool/.gitignore
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
poetry.lock |
5 changes: 5 additions & 0 deletions
5
tabular-templates/python-template/awesome-function-tool/CHANGELOG.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# CHANGELOG | ||
|
||
## 0.1.0 | ||
|
||
Initial release. |
26 changes: 26 additions & 0 deletions
26
tabular-templates/python-template/awesome-function-tool/Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
FROM polusai/bfio:2.1.9 | ||
|
||
# environment variables defined in polusai/bfio | ||
# ENV EXEC_DIR="/opt/executables" | ||
# ENV DATA_DIR="/data" | ||
# ENV POLUS_EXT=".ome.tif" | ||
# Change to WARNING for fewer logs, and DEBUG for debugging | ||
ENV POLUS_LOG="INFO" | ||
|
||
ENV POLUS_IMG_EXT=".ome.tif" | ||
ENV POLUS_TAB_EXT=".csv" | ||
|
||
# Work directory defined in the base container | ||
# WORKDIR ${EXEC_DIR} | ||
|
||
COPY pyproject.toml ${EXEC_DIR} | ||
COPY VERSION ${EXEC_DIR} | ||
COPY README.md ${EXEC_DIR} | ||
COPY CHANGELOG.md ${EXEC_DIR} | ||
COPY src ${EXEC_DIR}/src | ||
|
||
RUN pip3 install ${EXEC_DIR} --no-cache-dir | ||
|
||
# Default command. Additional arguments are provided through the command line | ||
ENTRYPOINT ["python3", "-m", "polus.tabular.package1.package2.awesome_function"] | ||
CMD ["--help"] |
23 changes: 23 additions & 0 deletions
23
tabular-templates/python-template/awesome-function-tool/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
# Awesome Tool (0.1.0) | ||
|
||
An awesome function. | ||
|
||
## Building | ||
|
||
To build the Docker image for the conversion plugin, run `./build-docker.sh`. | ||
|
||
## Install WIPP Plugin | ||
|
||
If WIPP is running, navigate to the plugins page and add a new plugin. Paste the | ||
contents of `plugin.json` into the pop-up window and submit. | ||
|
||
## Options | ||
|
||
This plugin takes 2 input arguments and 1 output argument: | ||
|
||
| Name | Description | I/O | Type | Default | ||
|---------------|-------------------------|--------|--------| | ||
| inpDir | Input image collection to be processed by this plugin | Input | collection | ||
| filePattern | Filename pattern used to separate data | Input | string | .* | ||
| preview | Generate an output preview | Input | boolean | False | ||
| outDir | Output collection | Output | collection |
1 change: 1 addition & 0 deletions
1
tabular-templates/python-template/awesome-function-tool/VERSION
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
0.1.0 |
4 changes: 4 additions & 0 deletions
4
tabular-templates/python-template/awesome-function-tool/build-docker.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
#!/bin/bash | ||
|
||
version=$(<VERSION) | ||
docker build . -t polusai/awesome-function-tool:${version} |
67 changes: 67 additions & 0 deletions
67
tabular-templates/python-template/awesome-function-tool/plugin.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
{ | ||
"name": "Awesome Tool", | ||
"version": "0.1.0", | ||
"title": "Awesome Tool", | ||
"description": "An awesome function.", | ||
"author": "Data Scientist ([email protected])", | ||
"institution": "National Center for Advancing Translational Sciences, National Institutes of Health", | ||
"repository": "https://github.com/polusAI/tabular-tools", | ||
"website": "https://ncats.nih.gov/preclinical/core/informatics", | ||
"citation": "", | ||
"containerId": "polusai/awesome-function-tool:0.1.0", | ||
"baseCommand": [ | ||
"python3", | ||
"-m", | ||
"polus.tabular.package1.package2.awesome_function" | ||
], | ||
"inputs": { | ||
"inpDir": { | ||
"type": "collection", | ||
"title": "Input collection", | ||
"description": "Input image collection to be processed by this plugin.", | ||
"required": "True" | ||
}, | ||
"filePattern": { | ||
"type": "string", | ||
"title": "Filename pattern", | ||
"description": "Filename pattern used to separate data.", | ||
"required": "False", | ||
"default": ".*" | ||
}, | ||
"preview": { | ||
"type": "boolean", | ||
"title": "Preview", | ||
"description": "Generate an output preview.", | ||
"required": "False", | ||
"default": "False" | ||
} | ||
}, | ||
"outputs": { | ||
"outDir": { | ||
"type": "collection", | ||
"description": "Output collection." | ||
} | ||
}, | ||
"ui": { | ||
"inpDir": { | ||
"type": "collection", | ||
"title": "Input collection", | ||
"description": "Input image collection to be processed by this plugin.", | ||
"required": "True" | ||
}, | ||
"filePattern": { | ||
"type": "string", | ||
"title": "Filename pattern", | ||
"description": "Filename pattern used to separate data.", | ||
"required": "False", | ||
"default": "False" | ||
}, | ||
"preview": { | ||
"type": "boolean", | ||
"title": "Filename pattern", | ||
"description": "Generate an output preview.", | ||
"required": "False", | ||
"default": "False" | ||
} | ||
} | ||
} |
32 changes: 32 additions & 0 deletions
32
tabular-templates/python-template/awesome-function-tool/pyproject.toml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
[tool.poetry] | ||
name = "polus-tabular-package1-package2-awesome-function" | ||
version = "0.1.0" | ||
description = "An awesome function." | ||
authors = ["Data Scientist <[email protected]>"] | ||
readme = "README.md" | ||
packages = [{include = "polus", from = "src"}] | ||
|
||
[tool.poetry.dependencies] | ||
python = ">=3.9,<3.12" | ||
bfio = {version = ">=2.3.3,<3.0", extras = ["all"]} | ||
filepattern = ">=2.0.4,<3.0" | ||
preadator = "^0.4.0.dev2" | ||
typer = "^0.7.0" | ||
|
||
[tool.poetry.group.dev.dependencies] | ||
bump2version = "^1.0.1" | ||
pytest = "^7.4" | ||
pytest-sugar = "^0.9.6" | ||
pre-commit = "^3.2.1" | ||
black = "^23.3.0" | ||
mypy = "^1.1.1" | ||
ruff = "^0.0.270" | ||
|
||
[build-system] | ||
requires = ["poetry-core"] | ||
build-backend = "poetry.core.masonry.api" | ||
|
||
[tool.pytest.ini_options] | ||
pythonpath = [ | ||
"." | ||
] |
20 changes: 20 additions & 0 deletions
20
tabular-templates/python-template/awesome-function-tool/run-plugin.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
#!/bin/bash | ||
|
||
version=$(<VERSION) | ||
|
||
# Update with your data | ||
inpDir=/tmp/path/to/input | ||
outDir=/tmp/path/to/output | ||
filepattern="pattern" | ||
|
||
|
||
container_input_dir="/inpDir" | ||
container_output_dir="/outDir" | ||
|
||
docker run -v $inpDir:/${container_input_dir} \ | ||
-v $outDir:/${container_output_dir} \ | ||
--user $(id -u):$(id -g) \ | ||
polusai/awesome-function-tool:${version} \ | ||
--inpDir ${container_input_dir} \ | ||
--filePattern ${filepattern} \ | ||
--outDir ${container_output_dir} |
7 changes: 7 additions & 0 deletions
7
...te/awesome-function-tool/src/polus/tabular/package1/package2/awesome_function/__init__.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
"""awesome_function.""" | ||
|
||
__version__ = "0.1.0" | ||
|
||
from polus.tabular.package1.package2.awesome_function.awesome_function import ( # noqa # pylint: disable=unused-import | ||
awesome_function, | ||
) |
Oops, something went wrong.