Skip to content

Preload geocoder database, to minimize timeouts#5603

Merged
zachmargolis merged 2 commits intomainfrom
margolis-preload-geocoder-database
Nov 12, 2021
Merged

Preload geocoder database, to minimize timeouts#5603
zachmargolis merged 2 commits intomainfrom
margolis-preload-geocoder-database

Conversation

@zachmargolis
Copy link
Contributor

@zachmargolis zachmargolis commented Nov 10, 2021

We see a few occasional geocoder timeouts in prod while reading from disk, my hypothesis was that the Geocoder gem lazily loads its datasource.

In a fresh Rails console in INT, I believe I confirmed this. The first lookup is ~0.1 seconds, and subsequent lookups are at least 200x faster, no matter the order or values, so I am pretty sure the first one forces a lookup

> timing = %w[0.0.0.0 1.2.3.4 8.8.8.8 192.168.1.1].map { |ip| Benchmark.realtime { Geocoder.search(ip) } }
=> [0.08297044999926584, 0.00040709099994273856, 0.0003728679985215422, 0.00019449899991741404]
timing[1..].map { |x| timing.first / x }
=> [203.81302954606332, 222.51963249260257, 426.58548390735075]

The first commit (c4e57b5) was just a personal preference of trying to have "positive" conditions first, which I think reads better.

The "meat" of this PR is 6d2a310

file: GEO_DATA_FILEPATH,
},
)
Geocoder.search('1.2.3.4') # the datasource is lazily loaded, make sure it eager loads
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this line is the meaningful change in here (6d2a310)

@zachmargolis zachmargolis merged commit 31248c6 into main Nov 12, 2021
@zachmargolis zachmargolis deleted the margolis-preload-geocoder-database branch November 12, 2021 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants