kickstarting python lib

[kpwebb]

@migurski just moving this thread over from sharedstreets/sharedstreets-ref-system#16 From the ticket:

So the SharedStreets tiles use length delimited encoding (the technique $GOOG recommends for files over ~1mb to allow stream processing of data). Turns out the Python Protobuf library doesn't have helper functions to parse these messages, but I found this example on how do to it:

https://www.datadoghq.com/blog/engineering/protobuf-parsing-in-python/

Just posted a quick example here using your compiled version of the .proto file:

https://github.com/sharedstreets/sharedstreets-python/blob/master/test.py

Let's use your example code as starting point for building tile loading functionality in the python lib.

Also @DenisCarriere want to create a punch list based on the core functionality from the JS lib so we can try and keep parity in the python version?

[migurski]

I tried out the test code, and it worked well. Should be possible to run with this.

[DenisCarriere]

to create a punch list based on the core functionality from the JS lib so we can try and keep parity in the python version

👍 Sounds good.

I'll try to build up the documentation to have both Python & Javascript sync up.

A few method names might need to change to follow the Python idioms.

Ex: forwardReference => forward_reference

Should we also divide the Python "core" library with the "pbf" parser (eventually writer).

• sharedstreets
• sharedstreets-pbf

[migurski]

I’d keep them together; there’s little benefit to prematurely carving up libraries just yet!

A few things I’d like to see in this library, that I’m willing to personally work on:

• Parsing the hosted PBF files into Python data structures
• Unified GeoJSON output
• Basic tile server with slippy map preview so people can see it working

[kpwebb]

@migurski this sounds great! and big +1 on the tile server with slippy preview. Having a CLI we can point at local tile directory would be fantastic.

I'm curious if we can leverage any of the the existing React UI (http://platform.sharedstreets.io/) to boostrap this. I'm about to flip the switch on the public repo of this. The extensible layer infrastructure still needs a lot of work but hopefully a good starting point.

[migurski]

What would the local tile directory CLI do? Fill it up, or convert it out?

[kpwebb]

I was thinking something like the node "serve" command to spin up a slippy map viewer for local files.