Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compress large read only table #20

Open
si-the-pie opened this issue Jan 18, 2019 · 0 comments
Open

Compress large read only table #20

si-the-pie opened this issue Jan 18, 2019 · 0 comments
Labels
build Main cyclestreets routing edition build a.k.a import

Comments

@si-the-pie
Copy link
Member

The map_leg_detail table describes the edges in the routing graph. Some of these are CSV fields (e.g. distances and elevations) which compress well.

The following technique (based on this) has been manually applied to the current routing edition, reducing the table size from 13G to just under 6G. After the compression procedure the table works in the same way as previously. The reduced data size should yield a modest performance boost, although no benchmarking has been performed.

This procedure could be added to the main build process, but is a bit fiddly as involves working as root user on the command line, possibly while the mysql service is stopped.

# [email protected]:#
cd /var/lib/mysql/routing190109

# File sizes before compression (data for routing190101 is 13G)
ls -lh map_leg_detail.*

# Check status
myisamchk -dvv map_leg_detail

# Pack (takes five minutes)
date
myisampack map_leg_detail
date

# Check compressed tables (takes three minutes)
date
myisamchk -rq map_leg_detail
date

# File sizes after compression
ls -lh map_leg_detail.*
# Data file is 5.9G (before was 13G)

# Flush tables
mysql -uroot -e "flush tables;"
@si-the-pie si-the-pie added the build Main cyclestreets routing edition build a.k.a import label Jan 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Main cyclestreets routing edition build a.k.a import
Projects
None yet
Development

No branches or pull requests

1 participant