Add PostgreSQL-independent way for testing imports from Nominatim #515
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The framework uses an H2 in-memory database to provide the data. Given that we only use simple SELECT statements, this is sufficiently close to the actual usage. The only change required is for hstore and geometry datatypes. hstore gets replaced by json. For geometry use H2's own geometry types. Both types need a slightly different form of marshalling. This is done by mocking the functions in DBUtils. I haven't found a usable way to replace the static functions, so DBUtils is now an instantiated class with member functions.
These tests depend on a correctly lodaed Nominatim database. For that reason they were never run and are possibly outdated by now because the necessary database cannot be recreated with today's OSM data.
More clean way of getting a different implementation into the tests. The code iteself includes an implementation for Postgis-Adaptation. The tests have an implementation for H2 which the Connector tests use. Allows to remove all mocking code.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR replaces the disabled import tests with unit tests for the NominatimConnector that implement the database accesses via a H2 in-memory database. The advantage of the H2 solution is that you don't need a PostgreSQL setup to run the tests and that the in-memory database is much faster to setup and tear down. Given that we only run relatively simple SELECT statements, the SQL differences are also minor enough to make the tests useful.
The only issue is with the hstore and geometry datatypes. I've replaced HSTORE with a simple JSON structure and the geometry with H2's own geometry type. That causes some trouble when demarshalling the data that comes from the database. The code for converting the returned values into Java datatypes is nicely encapsulated in DBUtils but I struggled to replace the functions for the tests. Providing my own implementation for the tests ended in 'Jar hell' when ES loads. Mocking static functions wasn't really working either. That is why there is now a DBUtils member in the NominatimConnector with non-static functions that can be mocked. Not pretty but it does the job. This doesn't need to be the final solution. If somebody with a better handle on Java can come up with a better solution, I'd warmly welcome a PR.
The first batch of tests only covers the basic functions of reading from placex, place_addressline and location_property_osmline. More complete tests to come in follow-up PRs.