-
Notifications
You must be signed in to change notification settings - Fork 102
Add FsspecJsonWSIReader class. #897
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
shaneahmed
merged 51 commits into
TissueImageAnalytics:develop
from
aacic:zarr-tiff-wsi-reader
Mar 7, 2025
Merged
Changes from 4 commits
Commits
Show all changes
51 commits
Select commit
Hold shift + click to select a range
1bc2356
Add ZarrTIFFWSIReader class.
aacic adb3574
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 1ce17b2
Merge branch 'develop' into zarr-tiff-wsi-reader
shaneahmed dc7c77c
Merge branch 'develop' into zarr-tiff-wsi-reader
shaneahmed 9276647
Merge branch 'develop' into zarr-tiff-wsi-reader
shaneahmed 0cf8c32
Rename ZarrTIFFWSIReader to FsspecJsonReader
aacic e750f2a
Rename ZarrTIFFWSIReader to FsspecJsonReader
aacic 0209100
Rename ZarrTIFFWSIReader to FsspecJsonWSIReader.
aacic 7b6a7b1
Rename ZarrTIFFWSIReader to FsspecJsonWSIReader.
aacic 2034262
Migrate tiff_fsspec.py.
aacic 91d5911
Migrate is_valid_zarr_fsspec.
aacic 224de85
Rename ZarrTIFFWSIReader to FsspecJsonWSIReader.
aacic 1bf88bc
Migrate is_valid_zarr_fsspec.
aacic e3d6129
Fix loggin issue.
aacic ad8ed13
Add Jpeg2k codec.
aacic ad559aa
WIP: Add DelegateWSIReader.
aacic afdf912
Merge branch 'develop' into zarr-tiff-wsi-reader
shaneahmed ebd41da
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 06b20f2
Update DelegateWSIReader
aacic c7b99bb
Update DelegateWSIReader
aacic 50f3267
Update DelegateWSIReader.
aacic 5c61c95
Update DelegateWSIReader.
aacic f320120
Update DelegateWSIReader.
aacic 7c92794
Remane DelegateWSIReader to TIFFWSIReaderDelegate.
aacic cea7c60
Add docs.
aacic 51d98e5
Add docs.
aacic 5e870a2
Extract parse_svs_metadata to Delagate class.
aacic 5923737
Fix test.
aacic 1f7d6bc
Fix tests.
aacic 1ce4610
Fix tests.
aacic 61c00e2
Register codecs
aacic 4b7860e
Add tests.
aacic b8dd291
Fix tests.
aacic 63aae0d
Fix metadata issue.
aacic d691939
Add test_fsspec_json_wsi_reader_instantiation test.
aacic 553cafa
Add no cover else branch.
aacic cdaa3ed
Add more tests.
aacic 6d5a372
Add more tests.
aacic 3bb9257
Add more tests.
aacic fd85e9a
Merge branch 'develop' into zarr-tiff-wsi-reader
aacic 6008f4c
Clean up tests.
aacic 4f3c1ad
Update docs.
aacic e902766
Update docs.
aacic 13b33ce
Update tiatoolbox/utils/tiff_to_fsspec.py
aacic c06c27e
Update tiatoolbox/wsicore/wsireader.py
aacic 57ac66d
Update tests/test_wsireader.py
aacic ee240bd
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 6317782
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 7e3ce79
Update docs.
aacic 4762d09
Merge branch 'develop' into zarr-tiff-wsi-reader
shaneahmed 3a78386
Merge branch 'develop' into zarr-tiff-wsi-reader
shaneahmed File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| """Test files meant for testing zarr tiff json files.""" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,147 @@ | ||
| """Module for processing SVS metadata and generating fsspec JSON file.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import json | ||
| import sys | ||
| from datetime import datetime | ||
| from pathlib import Path | ||
| from typing import TYPE_CHECKING, Any | ||
|
|
||
| from tifffile import TiffFile, TiffPages, tiff2fsspec | ||
|
|
||
| if TYPE_CHECKING: | ||
| from numbers import Number | ||
|
|
||
| # Constants | ||
| EXPECTED_KEY_VALUE_PAIRS = 2 | ||
| EXPECTED_ARG_COUNT = 4 | ||
| URL_PLACEHOLDER = "https://replace.me/" | ||
|
|
||
|
|
||
| def _parse_svs_metadata(pages: TiffPages) -> dict[str, Any]: | ||
| # Copy/paste from TIFFWSIReader._parse_svs_metadata, extract to the util method. | ||
| """Extract SVS-specific metadata.""" | ||
| raw = {} | ||
| mpp: list[float] | None = None | ||
| objective_power: float | None = None | ||
| vendor = "Aperio" | ||
|
|
||
| description = pages[0].description | ||
| raw["Description"] = description | ||
| parts = description.split("|") | ||
| description_headers, key_value_pairs = parts[0], parts[1:] | ||
| description_headers = description_headers.split(";") | ||
|
|
||
| software, photometric_info = description_headers[0].splitlines() | ||
| raw["Software"] = software | ||
| raw["Photometric Info"] = photometric_info | ||
|
|
||
| def parse_svs_tag(string: str) -> tuple[str, Number | str | datetime]: | ||
| """Parse SVS key-value string.""" | ||
| pair = string.split("=") | ||
| if len(pair) != EXPECTED_KEY_VALUE_PAIRS: | ||
| invalid_metadata_msg = ( | ||
| "Invalid metadata. Expected string of the format 'key=value'." | ||
| ) | ||
| raise ValueError(invalid_metadata_msg) | ||
|
|
||
| key, value_string = pair | ||
| key = key.strip() | ||
| value_string = value_string.strip() | ||
|
|
||
| def us_date(string: str) -> datetime: | ||
| """Return datetime parsed according to US date format.""" | ||
| return datetime.strptime(string, r"%m/%d/%y").astimezone() | ||
|
|
||
| def time(string: str) -> datetime: | ||
| """Return datetime parsed according to HMS format.""" | ||
| return datetime.strptime(string, r"%H:%M:%S").astimezone() | ||
|
|
||
| casting_precedence = [us_date, time, int, float] | ||
| value: Number | str | datetime = value_string | ||
| for cast in casting_precedence: | ||
| try: | ||
| value = cast(value_string) | ||
| break | ||
| except ValueError: | ||
| continue | ||
|
|
||
| return key, value | ||
|
|
||
| svs_tags = dict(parse_svs_tag(string) for string in key_value_pairs) | ||
| raw["SVS Tags"] = svs_tags | ||
| mpp = [svs_tags.get("MPP")] * 2 if svs_tags.get("MPP") is not None else None | ||
| objective_power = svs_tags.get("AppMag") | ||
|
|
||
| return { | ||
| "objective_power": objective_power, | ||
| "vendor": vendor, | ||
| "mpp": mpp, | ||
| "raw": raw, | ||
| } | ||
|
|
||
|
|
||
| def convert_metadata(metadata: dict) -> dict: | ||
| """Convert metadata to JSON-compatible format.""" | ||
| if isinstance(metadata, dict): | ||
| return {key: convert_metadata(value) for key, value in metadata.items()} | ||
| if isinstance(metadata, list): | ||
| return [convert_metadata(item) for item in metadata] | ||
| if isinstance(metadata, datetime): | ||
| return metadata.isoformat() # Convert datetime to ISO 8601 string | ||
| return metadata | ||
|
|
||
|
|
||
| def replace_url( | ||
| data: dict[str, Any], output_path: Path, old_url: str, new_url: str | ||
| ) -> None: | ||
| """Replace URL in the JSON file.""" | ||
| for value in data.values(): | ||
| if isinstance(value, list) and value[0] == old_url: | ||
| value[0] = new_url | ||
|
|
||
| with output_path.open("w") as json_file: | ||
| json.dump(data, json_file, indent=2) | ||
|
|
||
|
|
||
| def main(svs_file_path: str, json_file_path: str, final_url: str) -> None: | ||
| """Main function to handle SVS file processing.""" | ||
| url_to_replace = f"{URL_PLACEHOLDER}{Path(svs_file_path).name}" | ||
|
|
||
| tiff_file_pages = TiffFile(svs_file_path).pages | ||
|
|
||
| # Generate fsspec JSON | ||
| tiff2fsspec(svs_file_path, url=URL_PLACEHOLDER, out=json_file_path) | ||
|
|
||
| # Parse SVS metadata | ||
| metadata = _parse_svs_metadata(pages=tiff_file_pages) | ||
|
|
||
| # Convert metadata to JSON-compatible format | ||
| metadata_serializable = convert_metadata(metadata) | ||
|
|
||
| # Read the JSON data from the file | ||
| json_path = Path(json_file_path) | ||
| with json_path.open() as file: | ||
| json_data = json.load(file) | ||
|
|
||
| # Decode `.zattrs` JSON string into a dictionary | ||
| zattrs = json.loads(json_data[".zattrs"]) | ||
|
|
||
| # Update metadata into `.zattrs` | ||
| if "multiscales" in zattrs and isinstance(zattrs["multiscales"], list): | ||
| zattrs["multiscales"][0]["metadata"] = metadata_serializable | ||
|
|
||
| # Convert back to a JSON string | ||
| json_data[".zattrs"] = json.dumps(zattrs) | ||
|
|
||
| # Replace URLs in the JSON file | ||
| replace_url(json_data, json_path, url_to_replace, final_url) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| if len(sys.argv) != EXPECTED_ARG_COUNT: | ||
| msg = " Usage: python script.py <svs_file_path> <json_file_path> <final_url>" | ||
| raise ValueError(msg) | ||
|
|
||
| main(sys.argv[1], sys.argv[2], sys.argv[3]) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| """Module to run TileServer for testing purpose.""" | ||
|
|
||
| from flask_cors import CORS | ||
|
|
||
| from tiatoolbox.visualization import TileServer | ||
| from tiatoolbox.wsicore import WSIReader | ||
|
|
||
| svs = "/path/to/fsspec.json" | ||
|
|
||
| reader = WSIReader.open(svs) | ||
|
|
||
| # Initialize and run the TileServer | ||
| tile_server = TileServer( | ||
| title="Tiatoolbox TileServer", | ||
| layers={"layer": reader}, | ||
| ) | ||
| CORS(tile_server, send_wildcard=True) | ||
|
|
||
|
|
||
| tile_server.run(host="127.0.0.1", port=5000) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.