Skip to content

Commit

Permalink
added template for tabular tools
Browse files Browse the repository at this point in the history
  • Loading branch information
hamshkhawar committed Apr 9, 2024
1 parent 5519727 commit 27995c9
Show file tree
Hide file tree
Showing 46 changed files with 1,486 additions and 0 deletions.
Binary file added tabular-templates/.DS_Store
Binary file not shown.
1 change: 1 addition & 0 deletions tabular-templates/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
poetry.lock
23 changes: 23 additions & 0 deletions tabular-templates/python-template/.bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
[bumpversion]
current_version = 1.1.0-dev1
commit = False
tag = False
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\-(?P<release>[a-z]+)(?P<dev>\d+))?
serialize =
{major}.{minor}.{patch}-{release}{dev}
{major}.{minor}.{patch}

[bumpversion:part:release]
optional_value = _
first_value = dev
values =
dev
_

[bumpversion:part:dev]

[bumpversion:file:pyproject.toml]
search = version = "{current_version}"
replace = version = "{new_version}"

[bumpversion:file:README.md]
9 changes: 9 additions & 0 deletions tabular-templates/python-template/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# CHANGELOG

# 1.0.0

* Generate plugins from templates using cookiecutter.

# 1.1.0

* Generate plugins following updated [standard guidelines](https://labshare.atlassian.net/wiki/spaces/WIPP/pages/3275980801/Python+Plugin+Standards)
111 changes: 111 additions & 0 deletions tabular-templates/python-template/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# WIPP Plugin Cookie Cutter (for Python) (1.1.0-dev1)

This repository is a cookie cutter template that creates the basic scaffold structure of a
polus plugin and add it to the polus plugins directory structure.

## How to use
1. Clone `tools-templates` and change to the polus-plugins directory
2. `cd tabular-templates/python-template`
3. (optional) Install poetry if not available.
4. (optional) Create a dedicated environment with conda or venv.
5. Install the dependencies: `poetry install`
6. Modify `cookiecutter.json` to include author and plugin information.`plugin_package` should always start with `polus.tabular`.
** NOTE: ** Do not edit values in brackets ({}) as they are edited by cookiecutter directly.
Those are automatically generated from the previous entries. If your plugin is called
"Awesome Function", then the plugin folder and docker container will have the name `awesome-function-plugin`.
7. Create your plugin skeleton: `python -m cookiecutter . --no-input`


## Plugin Standard
The generated plugin will be compatible with polus most up-to-date guidelines :
see [standard guidelines](https://labshare.atlassian.net/wiki/spaces/WIPP/pages/3275980801/Python+Plugin+Standards)

The code generated provides out-of-box support for :
- customizing the plugin code.
- implementing tests.
- creating and running a container.
- managing versioning.
- updating documentation (README, CHANGELOG).
- maintaining a WIPP manifest (plugin.json).


## Executing the plugin

The plugin should be run as a package.
To install the package :

`pip install .`

The skeleton code can be run this way :
From the plugin's top directory (with the default values):

`python -m polus.tabular.package1.package2.awesome_function -i /tmp/inp -o /tmp/out`

This should print some logs with the provided inputs and outputs and return.

## Running tests
Plugin's developer should use `pytest`.
Some simple tests have been added to the template as examples.
Before submitting a PR to `tabular-tools`, other unit tests should be created and added to the `tests`
directory.

To run tests :

From the plugin's top directory, type `python -m pytest`.
Depending on how you have set up your environment, you may be able to run the pytest cli directly `pytest`. See pytest doc for how the project source directory is scanned to collect tests.
This should run a test successfully and return.


## Creating and running a container

` ./build-docker.sh && ./run-plugin.sh`

Build the docker image and run the container.

### DockerFile
A docker image is build from a base image with common dependencies pre-installed.
The image entrypoint will run the plugin's package entrypoint.

### build-docker.sh
Run this script to build the container.

### run-plugin.sh
Run the container locally.


## Customize the plugin

### Project code

A set of common dependencies are added to `pyproject.toml`.
Update according to your needs.

### Managing versioning

Making sure that the file version is consistent across files in a plugin can be
challenging, so the Python template now uses
[bump2version](https://github.com/c4urself/bump2version)
to help manage versioning. This automatically changes the `VERSION` and
`plugin.json` files to the next version, preventing you from having to remember
to change the version everywhere. The `bumpversion.cfg` can be modified to
change the version in other files as well.

To use this feature:
`bump2version --config-file bumpversion.cfg`

### Documentation

#### README.md

A basic description of what the plugin does. This should define all the inputs
and outputs.

#### CHANGELOG.md

Documents updates made to the plugin.


### WIPP manifest (plugin.json).

This file defines the input and output variables for WIPP, and defines the UI
components showed to the user.
1 change: 1 addition & 0 deletions tabular-templates/python-template/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.1.0-dev0
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
[bumpversion]
current_version = 0.1.0
commit = False
tag = False
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\-(?P<release>[a-z]+)(?P<dev>\d+))?
serialize =
{major}.{minor}.{patch}-{release}{dev}
{major}.{minor}.{patch}

[bumpversion:part:release]
optional_value = _
first_value = dev
values =
dev
_

[bumpversion:part:dev]

[bumpversion:file:pyproject.toml]
search = version = "{current_version}"
replace = version = "{new_version}"

[bumpversion:file:VERSION]

[bumpversion:file:README.md]

[bumpversion:file:plugin.json]

[bumpversion:file:src/polus/tabular/package1/package2/awesome_function/__init__.py]
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.venv
out
tests
__pycache__
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
poetry.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# CHANGELOG

## 0.1.0

Initial release.
26 changes: 26 additions & 0 deletions tabular-templates/python-template/awesome-function-tool/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
FROM polusai/bfio:2.1.9

# environment variables defined in polusai/bfio
# ENV EXEC_DIR="/opt/executables"
# ENV DATA_DIR="/data"
# ENV POLUS_EXT=".ome.tif"
# Change to WARNING for fewer logs, and DEBUG for debugging
ENV POLUS_LOG="INFO"

ENV POLUS_IMG_EXT=".ome.tif"
ENV POLUS_TAB_EXT=".csv"

# Work directory defined in the base container
# WORKDIR ${EXEC_DIR}

COPY pyproject.toml ${EXEC_DIR}
COPY VERSION ${EXEC_DIR}
COPY README.md ${EXEC_DIR}
COPY CHANGELOG.md ${EXEC_DIR}
COPY src ${EXEC_DIR}/src

RUN pip3 install ${EXEC_DIR} --no-cache-dir

# Default command. Additional arguments are provided through the command line
ENTRYPOINT ["python3", "-m", "polus.tabular.package1.package2.awesome_function"]
CMD ["--help"]
23 changes: 23 additions & 0 deletions tabular-templates/python-template/awesome-function-tool/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Awesome Tool (0.1.0)

An awesome function.

## Building

To build the Docker image for the conversion plugin, run `./build-docker.sh`.

## Install WIPP Plugin

If WIPP is running, navigate to the plugins page and add a new plugin. Paste the
contents of `plugin.json` into the pop-up window and submit.

## Options

This plugin takes 2 input arguments and 1 output argument:

| Name | Description | I/O | Type | Default
|---------------|-------------------------|--------|--------|
| inpDir | Input image collection to be processed by this plugin | Input | collection
| filePattern | Filename pattern used to separate data | Input | string | .*
| preview | Generate an output preview | Input | boolean | False
| outDir | Output collection | Output | collection
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0.1.0
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash

version=$(<VERSION)
docker build . -t polusai/awesome-function-tool:${version}
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
{
"name": "Awesome Tool",
"version": "0.1.0",
"title": "Awesome Tool",
"description": "An awesome function.",
"author": "Data Scientist ([email protected])",
"institution": "National Center for Advancing Translational Sciences, National Institutes of Health",
"repository": "https://github.com/polusAI/tabular-tools",
"website": "https://ncats.nih.gov/preclinical/core/informatics",
"citation": "",
"containerId": "polusai/awesome-function-tool:0.1.0",
"baseCommand": [
"python3",
"-m",
"polus.tabular.package1.package2.awesome_function"
],
"inputs": {
"inpDir": {
"type": "collection",
"title": "Input collection",
"description": "Input image collection to be processed by this plugin.",
"required": "True"
},
"filePattern": {
"type": "string",
"title": "Filename pattern",
"description": "Filename pattern used to separate data.",
"required": "False",
"default": ".*"
},
"preview": {
"type": "boolean",
"title": "Preview",
"description": "Generate an output preview.",
"required": "False",
"default": "False"
}
},
"outputs": {
"outDir": {
"type": "collection",
"description": "Output collection."
}
},
"ui": {
"inpDir": {
"type": "collection",
"title": "Input collection",
"description": "Input image collection to be processed by this plugin.",
"required": "True"
},
"filePattern": {
"type": "string",
"title": "Filename pattern",
"description": "Filename pattern used to separate data.",
"required": "False",
"default": "False"
},
"preview": {
"type": "boolean",
"title": "Filename pattern",
"description": "Generate an output preview.",
"required": "False",
"default": "False"
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
[tool.poetry]
name = "polus-tabular-package1-package2-awesome-function"
version = "0.1.0"
description = "An awesome function."
authors = ["Data Scientist <[email protected]>"]
readme = "README.md"
packages = [{include = "polus", from = "src"}]

[tool.poetry.dependencies]
python = ">=3.9,<3.12"
bfio = {version = ">=2.3.3,<3.0", extras = ["all"]}
filepattern = ">=2.0.4,<3.0"
preadator = "^0.4.0.dev2"
typer = "^0.7.0"

[tool.poetry.group.dev.dependencies]
bump2version = "^1.0.1"
pytest = "^7.4"
pytest-sugar = "^0.9.6"
pre-commit = "^3.2.1"
black = "^23.3.0"
mypy = "^1.1.1"
ruff = "^0.0.270"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[tool.pytest.ini_options]
pythonpath = [
"."
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/bin/bash

version=$(<VERSION)

# Update with your data
inpDir=/tmp/path/to/input
outDir=/tmp/path/to/output
filepattern="pattern"


container_input_dir="/inpDir"
container_output_dir="/outDir"

docker run -v $inpDir:/${container_input_dir} \
-v $outDir:/${container_output_dir} \
--user $(id -u):$(id -g) \
polusai/awesome-function-tool:${version} \
--inpDir ${container_input_dir} \
--filePattern ${filepattern} \
--outDir ${container_output_dir}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
"""awesome_function."""

__version__ = "0.1.0"

from polus.tabular.package1.package2.awesome_function.awesome_function import ( # noqa # pylint: disable=unused-import
awesome_function,
)
Loading

0 comments on commit 27995c9

Please sign in to comment.