-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Give option for setting tmp dir #15
Comments
@tomvantilburg yes -- we've been configuring this in our planet.pbf builds as it takes about 4TB of scratch disk space to process a global tile set. Flink doesn't make this obvious/easy to change but I'll post the steps we're using in the builder docs. That said, I'd love to know more about you're using the builder -- we've made a global z12 tile set that's hosted on S3 (we're in the process of moving it to the AWS public data sets program). I'm curious if that address your need or if you're doing something different with the app? |
Does the tileset involve simplification/aggregation? Or is it just about the bounds of every tile? |
Not sure I exactly follow. We're generating a complete tile set of OSM Planet as processed by the builder application (contains all the data described in https://github.com/sharedstreets/sharedstreets-ref-system) -- these references are used by the aggregation and conflation tools to link data. |
Sorry for the confusion, I was thinking too complicated with vector tiles in mind where you want different combinations of data on different zoom levels (like roadclass 6 and up are only included from z13 and up, or something like that). |
Currently fink uses the systems tmp dir for storing intermediate files.
It would be nice when the user can give a custom tmp dir for this so it would be possible to dedicate larger disks for the process.
I know there is an option in flink
taskmanager.tmp.dirs
but it is not clear how to set this.The text was updated successfully, but these errors were encountered: