Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Commit

Permalink
Add IceVision docs page (#677)
Browse files Browse the repository at this point in the history
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
  • Loading branch information
ethanwharris and mergify[bot] authored Aug 20, 2021
1 parent a154e51 commit 828fbf0
Show file tree
Hide file tree
Showing 4 changed files with 61 additions and 5 deletions.
15 changes: 13 additions & 2 deletions docs/source/api/core.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,15 +48,26 @@ _____________________
~flash.core.finetuning.NoFreeze
~flash.core.finetuning.UnfreezeMilestones

flash.core.integration.fiftyone
_______________________________
flash.core.integrations.fiftyone
________________________________

.. autosummary::
:toctree: generated/
:nosignatures:

~flash.core.integrations.fiftyone.utils.visualize

flash.core.integrations.icevision
_________________________________

.. autosummary::
:toctree: generated/
:nosignatures:

~flash.core.integrations.icevision.transforms.IceVisionTransformAdapter
~flash.core.integrations.icevision.transforms.default_transforms
~flash.core.integrations.icevision.transforms.train_default_transforms

flash.core.model
________________

Expand Down
1 change: 1 addition & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ Lightning Flash

integrations/providers
integrations/fiftyone
integrations/icevision

.. toctree::
:maxdepth: 1
Expand Down
44 changes: 44 additions & 0 deletions docs/source/integrations/icevision.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
.. _ice_vision:

#########
IceVision
#########

IceVision from airctic is an awesome computer vision framework which offers a curated collection of hundreds of high-quality pre-trained models for: object detection, keypoint detection, and instance segmentation.
In Flash, we've integrated the IceVision framework to provide: data loading, augmentation, backbones, and heads.
We use IceVision components in our: :ref:`object detection <object_detection>`, :ref:`instance segmentation <instance_segmentation>`, and :ref:`keypoint detection <keypoint_detection>` tasks.
Take a look at `their documentation <https://airctic.com/>`_ and star `IceVision on GitHub <https://github.com/airctic/IceVision>`_ to spread the open source love!

IceData
_______

The `IceData library <https://github.com/airctic/icedata>`_ is a community driven dataset hub for IceVision.
All of the datasets in IceData can be used out of the box with flash using our ``.from_folders`` methods and the ``parser`` argument.
Take a look at our :ref:`keypoint_detection` page for an example.

Albumentations with IceVision and Flash
_______________________________________

IceVision provides two utilities for using the `albumentations library <https://albumentations.ai/>`_ with their models:
- the ``Adapter`` helper class for adapting an any albumentations transform to work with IceVision records,
- the ``aug_tfms`` utility function that returns a standard augmentation recipe to get the most out of your model.

In Flash, we use the ``aug_tfms`` as default transforms for the: :ref:`object detection <object_detection>`, :ref:`instance segmentation <instance_segmentation>`, and :ref:`keypoint detection <keypoint_detection>` tasks.
You can also provide custom transforms from albumentations using the :class:`~flash.core.integrations.icevision.transforms.IceVisionTransformAdapter` (which relies on the IceVision ``Adapter`` underneath).
Here's an example:

.. code-block:: python
import albumentations as A
from flash.core.integrations.icevision.transforms import IceVisionTransformAdapter
from flash.image import ObjectDetectionData
train_transform = {
"pre_tensor_transform": IceVisionTransformAdapter([A.HorizontalFlip(), A.Normalize()]),
}
datamodule = ObjectDetectionData.from_coco(
...,
train_transform=train_transform,
)
6 changes: 3 additions & 3 deletions flash/core/integrations/icevision/transforms.py
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ def from_icevision_record(record: "BaseRecord"):
class IceVisionTransformAdapter(nn.Module):
def __init__(self, transform):
super().__init__()
self.transform = transform
self.transform = A.Adapter(transform)

def forward(self, x):
record = to_icevision_record(x)
Expand All @@ -186,13 +186,13 @@ def forward(self, x):
def default_transforms(image_size: Tuple[int, int]) -> Dict[str, Callable]:
"""The default transforms from IceVision."""
return {
"pre_tensor_transform": IceVisionTransformAdapter(A.Adapter([*A.resize_and_pad(image_size), A.Normalize()])),
"pre_tensor_transform": IceVisionTransformAdapter([*A.resize_and_pad(image_size), A.Normalize()]),
}


@requires_extras("image")
def train_default_transforms(image_size: Tuple[int, int]) -> Dict[str, Callable]:
"""The default augmentations from IceVision."""
return {
"pre_tensor_transform": IceVisionTransformAdapter(A.Adapter([*A.aug_tfms(size=image_size), A.Normalize()])),
"pre_tensor_transform": IceVisionTransformAdapter([*A.aug_tfms(size=image_size), A.Normalize()]),
}

0 comments on commit 828fbf0

Please sign in to comment.