Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated to latest GeoLiteCity Binary on June 2017. #46

Merged
merged 8 commits into from
Dec 18, 2017
Merged

Updated to latest GeoLiteCity Binary on June 2017. #46

merged 8 commits into from
Dec 18, 2017

Conversation

y-ken
Copy link
Owner

@y-ken y-ken commented Jun 18, 2017 •
edited
Loading

Changelog

  • Updated to current latest binary which has released on 2017-12-05.
    • GeoLiteCity.dat
    • GeoLite2-City.mmdb

Geolocate Response check

Binary as before bundled on Dec 2014.

  • filesize: 18.7MB
  • everything have city information.
$ geoiplookup -f data/GeoLiteCity-2014-Dec.dat 8.8.8.8
GeoIP City Edition, Rev 1: US, CA, California, Mountain View, N/A, 37.386002, -122.083801, 807, 650
$ geoiplookup -f data/GeoLiteCity-2014-Dec.dat 66.102.3.80
GeoIP City Edition, Rev 1: US, CA, California, Mountain View, 94043, 37.419201, -122.057404, 807, 650
$ geoiplookup -f data/GeoLiteCity-2014-Dec.dat 125.54.15.42
GeoIP City Edition, Rev 1: JP, 34, Saitama, Tokorozawa, N/A, 35.786400, 139.467804, 0, 0

Binary on June 2017.

  • GeoLiteCity filesize: 19.1MB
  • GeoLite2-City filesize: 53.8MB
  • 8.8.8.8 has only country
  • 125.54.15.42 has postal code
$ geoiplookup -f data/GeoLiteCity-2017-Jun.dat 8.8.8.8
GeoIP City Edition, Rev 1: US, N/A, N/A, N/A, N/A, 37.750999, -97.821999, 0, 0
$ geoiplookup -f data/GeoLiteCity-2017-Jun.dat 66.102.3.80
GeoIP City Edition, Rev 1: US, CA, California, Mountain View, 94043, 37.419201, -122.057404, 807, 650
$ geoiplookup -f data/GeoLiteCity-2017-Jun.dat 125.54.15.42
GeoIP City Edition, Rev 1: JP, 34, Saitama, Tokorozawa, 359-1126, 35.786400, 139.467804, 0, 0

Binary on Dec 2017.

  • GeoLiteCity filesize: 17.7MB
  • GeoLite2-City filesize: 63.5MB
  • 125.54.15.42 has updated postal code
$ geoiplookup -f data/GeoLiteCity-2017-Dec.dat 8.8.8.8
GeoIP City Edition, Rev 1: US, N/A, N/A, N/A, N/A, 37.750999, -97.821999, 0, 0
$ geoiplookup -f data/GeoLiteCity-2017-Dec.dat 66.102.3.80
GeoIP City Edition, Rev 1: US, CA, California, Mountain View, 94043, 37.419201, -122.057404, 807, 650
$ geoiplookup -f data/GeoLiteCity-2017-Dec.dat 125.54.15.42
GeoIP City Edition, Rev 1: JP, 34, Saitama, Tokorozawa, 359-0036, 35.786400, 139.467804, 0, 0

@okkez okkez mentioned this pull request Dec 13, 2017
Merged
y-ken added 2 commits December 17, 2017 17:07
@y-ken
#46 Updated to GeoLiteCity, GeoLite2-City Binary on Dec 2017.
665aa3f 
* GeoLiteCity filesize: 17.7MB
* GeoLite2-City filesize: 63.5MB
* 125.54.15.42 has updated postal code

$ geoiplookup -f data/GeoLiteCity-2017-Dec.dat 8.8.8.8
GeoIP City Edition, Rev 1: US, N/A, N/A, N/A, N/A, 37.750999,
-97.821999, 0, 0
$ geoiplookup -f data/GeoLiteCity-2017-Dec.dat 66.102.3.80
GeoIP City Edition, Rev 1: US, CA, California, Mountain View, 94043,
37.419201, -122.057404, 807, 650
$ geoiplookup -f data/GeoLiteCity-2017-Dec.dat 125.54.15.42
GeoIP City Edition, Rev 1: JP, 34, Saitama, Tokorozawa, 359-0036,
35.786400, 139.467804, 0, 0
@y-ken
Changed testing ip address
bc10c63 
changed test ip add from 8.8.8.8 to 66.102.3.80
@y-ken
Copy link
Owner Author

y-ken commented Dec 17, 2017

@okkez
I'd like to release final version for Fluentd v0.12 with newest geoip city database.
But I faced partial test failing of older ruby version.
I'm afraid Would you please help me to fix it?
https://travis-ci.org/y-ken/fluent-plugin-geoip/builds/317612662

@okkez
Copy link
Collaborator

okkez commented Dec 18, 2017

@y-ken Hmm, it is caused by ruby_dig's bug.
Fluentd v1.0.1 depends ruby_dig for Ruby 2.2 and Ruby 2.1.
But ruby_dig v0.0.2 has a bug that it cannot delegate dig method to GeoIP2::LookupResult.

@okkez
Prefer backport_dig gem than dig_rb gem
6c47f41 
Because backport_dig gem is faster than dig_rb gem and Fluentd
v1.0.1 (record_accessor) uses backport_dig gem.
@y-ken
Copy link
Owner Author

y-ken commented Dec 18, 2017 •
edited
Loading

Oh, sounds great to use backport_dig instead of dig_rb!
I have been thought that I need to convert dig_rb friendry hash.

@okkez
Copy link
Collaborator

okkez commented Dec 18, 2017

I've added a commit to fix the problem.
I will report the ruby_dig's problem to Fluentd later.

@y-ken
Copy link
Owner Author

y-ken commented Dec 18, 2017 •
edited
Loading

I have been confused of this problem but I got workaround by @okkez .
Thank you very much!

@y-ken
Copy link
Owner Author

y-ken commented Dec 18, 2017

It seems tests failed at specific version of ruby..

@okkez
Copy link
Collaborator

okkez commented Dec 18, 2017

I'm checking failures.
backport_dig does not support Ruby 2.3 or later. So we cannot add backport_dig gem to spec.runtime_depencency...

@y-ken
Copy link
Owner Author

y-ken commented Dec 18, 2017

How about implement backport_dig to noop for ruby 2.3 or later that it acts like ruby_dig.
because ruby_dig description has It is a no-op if you are running Ruby 2.3 or greater. message.
https://github.com/Invoca/ruby_dig

@okkez
Copy link
Collaborator

okkez commented Dec 18, 2017

I've fixed problems related to #dig method and passed all tests.

@y-ken
Copy link
Owner Author

y-ken commented Dec 18, 2017 •
edited
Loading

Thanks!
It is just what i want to know, It seems master branch of fluentd still using ruby_dig.
https://github.com/fluent/fluentd/blob/master/fluentd.gemspec#L31
When does it enable backport_dig for fluentd? td-agent3?
https://github.com/fluent/fluentd/blob/4ccb48580bee3f91ea8d6ab0dbfdced219f1897a/lib/fluent/plugin_helper/record_accessor.rb#L20

@okkez
Copy link
Collaborator

okkez commented Dec 18, 2017

When does it enable backport_dig for fluentd? td-agent3?

Since Fluentd v0.14.20 and td-agent 3.1.0 (it bundles fluentd v0.14.25).

@okkez
Copy link
Collaborator

okkez commented Dec 18, 2017

See also fluent/fluentd#1794

@y-ken
Copy link
Owner Author

y-ken commented Dec 18, 2017

Would you please merge and release fluent-plugin-geoip v0.8.0?

@okkez okkez merged commit e832325 into master Dec 18, 2017
@okkez okkez deleted the update-geoip-database branch December 18, 2017 08:33
@okkez
Copy link
Collaborator

okkez commented Dec 18, 2017

released v0.8.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@y-ken @okkez