Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redesign PatchWSIDataset #4152

Merged
merged 17 commits into from
Apr 22, 2022
Merged

Redesign PatchWSIDataset #4152

merged 17 commits into from
Apr 22, 2022

Conversation

drbeh
Copy link
Member

@drbeh drbeh commented Apr 20, 2022

Fixes #3930

Description

This PR refactor PatchWSIDataset and uses new WSIReader as the backend.

Status

Ready

Types of changes

  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

Signed-off-by: Behrooz <[email protected]>
Signed-off-by: Behrooz <[email protected]>
Signed-off-by: Behrooz <[email protected]>
Signed-off-by: Behrooz <[email protected]>
@drbeh drbeh requested review from Nic-Ma and wyli April 20, 2022 18:37
monai/data/wsi_datasets.py Outdated Show resolved Hide resolved
monai/data/wsi_datasets.py Outdated Show resolved Hide resolved
monai/data/wsi_datasets.py Outdated Show resolved Hide resolved
monai/data/wsi_datasets.py Show resolved Hide resolved
monai/data/wsi_datasets.py Outdated Show resolved Hide resolved
@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 21, 2022

Hi @drbeh ,

BTW, about the SmartCachePatchWSIDataset, I think your previous suggestion was correct, we can slightly optimize the CacheDataset and SmartCacheDataset, etc. Then we can use them as decorators on another dataset object in future.

For now, you can use PatchWSIDataset as the data arg of SmartCacheDataset in your program to replace:
https://github.com/Project-MONAI/MONAI/blob/dev/monai/apps/pathology/data/datasets.py#L166

We must implement SmartCachePatchWSIDataset before because Clara 4.0 only supports 1 dataset in the config, now it's not necessary anymore.

Thanks.

Signed-off-by: Behrooz <[email protected]>
@drbeh
Copy link
Member Author

drbeh commented Apr 21, 2022

Hi @drbeh ,

BTW, about the SmartCachePatchWSIDataset, I think your previous suggestion was correct, we can slightly optimize the CacheDataset and SmartCacheDataset, etc. Then we can use them as decorators on another dataset object in future.

For now, you can use PatchWSIDataset as the data arg of SmartCacheDataset in your program to replace: https://github.com/Project-MONAI/MONAI/blob/dev/monai/apps/pathology/data/datasets.py#L166

We must implement SmartCachePatchWSIDataset before because Clara 4.0 only supports 1 dataset in the config.

Thanks.

Sure, will add SmartCache in a seperate PR as the test cases need some attention.

@drbeh
Copy link
Member Author

drbeh commented Apr 22, 2022

Hi @Nic-Ma, any other comment here? thanks :)

Signed-off-by: Behrooz <[email protected]>
Signed-off-by: Behrooz <[email protected]>
Copy link
Contributor

@Nic-Ma Nic-Ma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick update.
Looks good to me.

monai/data/wsi_datasets.py Show resolved Hide resolved
@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 22, 2022

/black

@Nic-Ma
Copy link
Contributor

Nic-Ma commented Apr 22, 2022

/build

@drbeh drbeh enabled auto-merge (squash) April 22, 2022 13:56
@drbeh drbeh merged commit 65d7296 into Project-MONAI:dev Apr 22, 2022
@drbeh drbeh deleted the new-wsi-datasets branch May 2, 2022 15:39
Can-Zhao pushed a commit to Can-Zhao/MONAI that referenced this pull request May 10, 2022
* Implement PatchWSIDataset

Signed-off-by: Behrooz <[email protected]>

* Add unittests

Signed-off-by: Behrooz <[email protected]>

* Add docs

Signed-off-by: Behrooz <[email protected]>

* Reorder imports

Signed-off-by: Behrooz <[email protected]>

* formatting:

Signed-off-by: Behrooz <[email protected]>

* Address comments

Signed-off-by: Behrooz <[email protected]>

* Update to be compatible with Dataset

Signed-off-by: Behrooz <[email protected]>

* Update reader to accept str, class, object

Signed-off-by: Behrooz <[email protected]>

* Add test cases for various reader and level arguments

Signed-off-by: Behrooz <[email protected]>

* Update comment about OpenSlide cache

Signed-off-by: Behrooz <[email protected]>

* Rename reader_name to backend

Signed-off-by: Behrooz <[email protected]>

* Add new test cases

Signed-off-by: Behrooz <[email protected]>

* Add unittests for openslide

Signed-off-by: Behrooz <[email protected]>

* Add new test cases

Signed-off-by: Behrooz <[email protected]>

* sorts

Signed-off-by: Behrooz <[email protected]>

* Add docstring for kwargs

Signed-off-by: Behrooz <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor PatchWSIDataset
2 participants