Create new endpoint for geometries of cities

[Joxit]

Hi,

A new endpoint could be interesting.
An endpoint that would send the geometry of a city using Who's On First ID.
Thanks to this we could highlight cities (like Who's On First spelunker or Google).
This could be another microservice (like pip-service or a new one) and should only serve localities/localadmins (because the regions/countries will be too large).
The answer should also be filtered, do not send the raw data from WOF (keep proprieties about geometry and bbox for example).

Example of request with properties:

GET /v1/place/geom?id=101751119 HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "id": 101751119,
  "type": "Feature",
  "properties": {
    "geom:area":0.012915,
    "geom:area_square_m":105073572.071737,
    "geom:bbox":"2.22407844723,48.8155766678,2.46976095835,48.9021619073",
    "geom:latitude":48.856626,
    "geom:longitude":2.342883,
    "src:geom":"mz",
    "wof:country":"FR",
    "wof:geomhash":"bcec769e4dc183e582c6cd5dbfd4de9a",
    "wof:id":101751119,
    "wof:lang":[
        "fre"
    ],
    "wof:name":"Paris",
    "wof:placetype":"locality",
},
  "bbox": [
    2.224078447233478,
    48.81557666781795,
    2.469760958345428,
    48.90216190732557
],
  "geometry": {"coordinates":[[],[]],"type":"MultiPolygon"}
}

[orangejulius]

Yes we definitely want this. We've been talking about it for a long time. The entire point of the /v1/place endpoint is to return more details, such as geometry, but we never got around to implementing it.

The venicegeo team has been working on importing geometries into Elasticsearch as part of WOF records. I don't know if they've done a full planet import, in which case they have not had to worry about the 100MB New Zealand polygon. I like your idea of not returning countries, since those geometries do get quite large. We could also create simplified geometries for display in larger cases.

[missinglink]

It's an interesting idea and a few people have asked for it.

In the past, we've stayed away from providing information for display, visual polygons weren't considered part of the 'core geocoding experience'.
On the other hand, it kind of makes sense, as we have all the data anyway so the feature wouldn't be super difficult to build.

An important consideration is that the API would need to be capable of performing douglas peucker style simplification on the polygons, some are very large (100's of MB) and wouldn't be suitable for display on mobile devices.

The simplification algorithm is CPU-bound and would be a heavy burden on the machines serving it.
An alternative solution would be to pre-simplify the geometries and store them in some sort of cache.

I personally feel that when these things are considered, another technology such as vector tiles is more suited to this job.
They would further expand on this API by adding a 'tiled' strategy, allowing a leaflet-style map to render tesselated polygons as they loaded.

We'd need to think a little more about which service features were going to be available and what the scope of the project would be.

[Joxit]

@orangejulius Oh yes, venicegeo is doing it in a simple way. It may be a good idea to integrate it into Elasticsearch instead of having filesystem only like pip-service.
Countries are large and will use CPU + bandwidth (ES to pelias-api to Client), this will slow down requests, that's why I think we can drop them.

@missinglink That's right, this is not really related to geocoding but is a cool feature (and can provide a wow effect).
I don't know if it's the API which should simplify the polygon or WOF importer (like venicegeo). I think we can directly store the simplified version because the raw geometry will never be sent (because it's not the first goal).

[Joxit]

I thought of something else, if data are stored in elasticsearch, you have to remember that the geometries should not be sent when we have search/autocomplete queries like this one: /v1/search?text=Paris&sources=wof.
I do not know if elasticsearch can filter the documents before sending a response. If that's not possible it will be the API that will do it, but it will slow down the queries.

[orangejulius]

Yes @Joxit that's a very good point. The overhead of sending the geometry JSON to the API with each request would be huge.

Fortunately it looks like it's possible to return only certain fields. It looks like it's different in Elasticsearch 5 vs 2 though, so we should watch out for that.

[Joxit]

That's cool, there is also an exclude source 😄

Here is a preview of this issue using whosonfirst-data/whosonfirst-data repository.