Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Give option for setting tmp dir #15

Open
tomvantilburg opened this issue Apr 30, 2018 · 4 comments
Open

Give option for setting tmp dir #15

tomvantilburg opened this issue Apr 30, 2018 · 4 comments

Comments

@tomvantilburg
Copy link

Currently fink uses the systems tmp dir for storing intermediate files.
It would be nice when the user can give a custom tmp dir for this so it would be possible to dedicate larger disks for the process.

I know there is an option in flink taskmanager.tmp.dirs but it is not clear how to set this.

@kpwebb
Copy link
Member

kpwebb commented Apr 30, 2018

@tomvantilburg yes -- we've been configuring this in our planet.pbf builds as it takes about 4TB of scratch disk space to process a global tile set. Flink doesn't make this obvious/easy to change but I'll post the steps we're using in the builder docs.

That said, I'd love to know more about you're using the builder -- we've made a global z12 tile set that's hosted on S3 (we're in the process of moving it to the AWS public data sets program). I'm curious if that address your need or if you're doing something different with the app?

@tomvantilburg
Copy link
Author

Does the tileset involve simplification/aggregation? Or is it just about the bounds of every tile?
Would definitely be interested in a service, but for the rest just playing around.

@kpwebb
Copy link
Member

kpwebb commented May 1, 2018

Not sure I exactly follow. We're generating a complete tile set of OSM Planet as processed by the builder application (contains all the data described in https://github.com/sharedstreets/sharedstreets-ref-system) -- these references are used by the aggregation and conflation tools to link data.

@tomvantilburg
Copy link
Author

Sorry for the confusion, I was thinking too complicated with vector tiles in mind where you want different combinations of data on different zoom levels (like roadclass 6 and up are only included from z13 and up, or something like that).
Having this complete tile set available to play around with would be great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants