Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: rename tiff file if not aligned with its tile name TDE-497 #209

Merged
merged 7 commits into from
Nov 17, 2022

Conversation

paulfouquet
Copy link
Collaborator

@paulfouquet paulfouquet commented Nov 8, 2022

Description

Some tiff files does not respect the naming convention. This functionality check the correct name against the tile name link to the tiff and rename the tiff file if the name is not correct.
The associated workflow in Argo has to be updated once this is merged.
non_visual_qa.py has been deleted so associated workflow has to be deleted when this PR is merged.

self.check_band_count(gdalinfo_result)
self.check_color_interpretation(gdalinfo_result)
def check_tile_and_rename(self, gdalinfo: Dict[Any, Any]) -> None:
origin = Point(gdalinfo["cornerCoordinates"]["upperLeft"][0], gdalinfo["cornerCoordinates"]["upperLeft"][1])
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check if the origin is valid

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two examples of files with origin "drift":
s3://linz-imagery-upload/Tasman Nelson Floods 2022 LiDAR & Orthos/renamed/top-of-the-south-flood/tifs/BP27_1000_4817.tif
s3://linz-imagery-upload/Invercargill2022_Pgrm3016/renamed/InvercargillUrban2022/tifs-RGB/CH10_500_001052.tif

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to work on the feature to correct the origin drift sooner rather than later otherwise files with origin drift will be renamed e.g. file name "BP27_1000_4817.tiff" with origin = Point(1643679.999967818148434, 5444159.999954843893647) will be renamed to BP27_1000_4816.tiff

Copy link
Collaborator Author

@paulfouquet paulfouquet Nov 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could create an extra check using a regex to check if the file name match the correct pattern ({tile_name}_{scale}_{originID}- like what is done in argo-task https://github.com/linz/argo-tasks/blob/master/src/commands/tileset-validate/tileset.validate.ts) AND <> than generated name, and logging or raising a specific issue for those cases?
@dwsilk

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If an origin coordinate is between coordinate - 0.01 < coordinate < coordinate + 0.01, e.g. less than 1cm difference from the origin coordinate we expect, then rename. Otherwise error due to an invalid origin.

MDavidson17
MDavidson17 previously approved these changes Nov 14, 2022
@amfage amfage self-requested a review November 15, 2022 21:20
amfage
amfage previously approved these changes Nov 15, 2022
Copy link
Contributor

@amfage amfage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one note inline about origin drift.

@paulfouquet paulfouquet marked this pull request as draft November 16, 2022 00:27
@paulfouquet paulfouquet dismissed stale reviews from amfage and MDavidson17 via d17f480 November 16, 2022 02:55
@paulfouquet paulfouquet marked this pull request as ready for review November 16, 2022 02:55
@paulfouquet paulfouquet marked this pull request as draft November 16, 2022 03:58
@paulfouquet paulfouquet reopened this Nov 16, 2022
@paulfouquet paulfouquet marked this pull request as ready for review November 16, 2022 21:49
amfage
amfage previously approved these changes Nov 16, 2022
@kodiakhq kodiakhq bot merged commit 7e547d6 into master Nov 17, 2022
@kodiakhq kodiakhq bot deleted the feat/rename-tiff-against-tile-index-tde-497 branch November 17, 2022 21:25
@github-actions github-actions bot mentioned this pull request Nov 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants