Harden some of the sources when trying to read bad files #1623
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was tested by running lisource_compare.py on most of the GDAL
repository's autotest files. If a source threw a traceback rather than
returning that it could not read a source, it was modified to handle the
condition better.
Specifically, command like `find ../gdal/autotest/{gcore,gdrivers} -type
f -not -name '*.py' -not -name '*.txt' -exec python
test/lisource_compare.py --all --projection= --projection=EPSG:3857 {}
\+` was run, though some test files were excluded.
manthey
force-pushed
the
harden-sources
branch
from
12820d5
to
364ca64
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This was tested by running lisource_compare.py on most of the GDAL repository's autotest files. If a source threw a traceback rather than returning that it could not read a source, it was modified to handle the condition better.
Specifically, command like
find ../gdal/autotest/{gcore,gdrivers} -type f -not -name '*.py' -not -name '*.txt' -exec python test/lisource_compare.py --all --projection= --projection=EPSG:3857 {} \+was run, though some test files were excluded.There are some interesting test cases that are just astonishingly slow. For instance, some test files are conceptually single layers that are 2e9 pixels across; we currently synthesize lower levels when getting things like thumbnails in some sources (such as tiff). We could be smarter there -- if the synthesized layer is far enough from the known parent, we could reduce the amount of work done by essentially compositing from single pixels (nearest neighbor) instead of tiles.
There are some other test cases that cause issues -- the geospatial pdfs cause bioformats to crash the jvm; when reprojected our build of gdal also has some uncaught error.