diff --git a/CHANGELOG.md b/CHANGELOG.md index 960f3d7..d335c52 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,6 @@ +## 7.2.2 + - [DOC] Add documentation for database auto-update behavior and database metrics [#187](https://github.com/logstash-plugins/logstash-filter-geoip/pull/187) + ## 7.2.1 - Republish the gem due to missing jars in 7.2.0 [#186](https://github.com/logstash-plugins/logstash-filter-geoip/pull/186) diff --git a/docs/index.asciidoc b/docs/index.asciidoc index 33bdb08..4376412 100644 --- a/docs/index.asciidoc +++ b/docs/index.asciidoc @@ -43,9 +43,7 @@ If you would like to get Autonomous System Number(ASN) information, you can use https://www.maxmind.com[MaxMind] changed from releasing the GeoIP database under a Creative Commons (CC) license to a proprietary end-user license agreement (EULA). The MaxMind EULA requires Logstash to update the MaxMind database -within 30 days of a database update. If Logstash fails to download the database -for 30 days, the geoip filter will stop enriching events in order to maintain compliance. -Events will be tagged with `_geoip_expired_database` tag to facilitate the handling of this situation. +within 30 days of a database update. The GeoIP filter plugin can manage the database for users running the Logstash default distribution, or you can manage @@ -57,6 +55,77 @@ Otherwise, you are responsible for maintaining compliance. The Logstash open source distribution uses the MaxMind Creative Commons license database by default. +[id="plugins-{type}s-{plugin}-database_auto"] +==== Database Auto-update + +This plugin bundles Creative Commons (CC) license databases. +In air-gapped environments, Logstash can use CC license databases indefinitely. +Logstash checks for database updates every day. It downloads the latest and can replace the old database +while the plugin is running. +After Logstash downloads EULA license databases, it will not fallback to CC license databases. + +If Logstash fails to download the database for 30 days, +the geoip filter will stop enriching events in order to maintain compliance. +Events will be tagged with `_geoip_expired_database` tag to facilitate the handling of this situation. + +TIP: When possible, allow Logstash to access the internet to download databases so that they are always up-to-date. + +[id="plugins-{type}s-{plugin}-metrics"] +==== Database Metrics + +You can monitor database status through the {logstash-ref}/node-stats-api.html#node-stats-api[Node Stats API]. + +The following request returns a JSON document containing database manager stats, +including: + +* database status and freshness +** `geoip_download_manager.database.*.status` +*** `init` : initial CC database status +*** `up_to_date` : using up-to-date EULA database +*** `to_be_expired` : 25 days without calling service +*** `expired` : 30 days without calling service +** `fail_check_in_days` : number of days Logstash fails to call service since the last success +* info about download successes and failures +** `geoip_download_manager.download_stats.successes` number of successful checks and downloads +** `geoip_download_manager.download_stats.failures` number of failed check or download +** `geoip_download_manager.download_stats.status` +*** `updating` : check and download at the moment +*** `succeeded` : last download succeed +*** `failed` : last download failed + +[source,js] +-------------------------------------------------- +curl -XGET 'localhost:9600/_node/stats/geoip_download_manager?pretty' +-------------------------------------------------- + +Example response: + +[source,js] +-------------------------------------------------- +{ + "geoip_download_manager" : { + "database" : { + "ASN" : { + "status" : "up_to_date", + "fail_check_in_days" : 0, + "last_updated_at": "2021-06-21T16:06:54+02:00" + }, + "City" : { + "status" : "up_to_date", + "fail_check_in_days" : 0, + "last_updated_at": "2021-06-21T16:06:54+02:00" + } + }, + "download_stats" : { + "successes" : 15, + "failures" : 1, + "last_checked_at" : "2021-06-21T16:07:03+02:00", + "status" : "succeeded" + } + } +} +-------------------------------------------------- + ==== Details A `[geoip][location]` field is created if diff --git a/logstash-filter-geoip.gemspec b/logstash-filter-geoip.gemspec index 2abcf12..032a31d 100644 --- a/logstash-filter-geoip.gemspec +++ b/logstash-filter-geoip.gemspec @@ -1,7 +1,7 @@ Gem::Specification.new do |s| s.name = 'logstash-filter-geoip' - s.version = '7.2.1' + s.version = '7.2.2' s.licenses = ['Apache License (2.0)'] s.summary = "Adds geographical information about an IP address" s.description = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program"