Affecting all Beats
Metricbeat
-
Fix Kubernetes overview dashboard views for non default time ranges. https://github.com/elastic/beats/issues/6395{6395}
Affecting all Beats
Auditbeat
Metricbeat
Auditbeat
-
Fixed an issue where the proctitle value was being truncated. 6080
-
Fixed an issue where values were incorrectly interpretted as hex data. 6080
-
Fixed parsing of the
keyvalue when multiple keys are present. 6080 -
Fix possible resource leak if file_integrity module is used with config reloading on Windows or Linux. 6198
Filebeat
-
Fix variable name for
convert_timezonein the system module. 5936
Metricbeat
-
Fix error
datastore '*' not foundin Vsphere module. 4879 -
Fix error
NotAuthenticatedin Vsphere module. 4673 -
Fix mongodb session consistency mode to allow command execution on secondary nodes. 4689
-
Fix kubernetes
state_podstatus.phaseso that the active phase is returned instead ofunknown. 5980 -
Fix error collecting network_names in Vsphere module. 5962
-
Fix process cgroup memory metrics for memsw, kmem, and kmem_tcp. 6033
-
Fix kafka OffsetFetch request missing topic and partition parameters. 5880
Packetbeat
-
Fix mysql SQL parser to trim
\rfrom Windows ServerSELECT\r\n\t1. 5572
Affecting all Beats
-
Adding a local keystore to allow user to obfuscate password 5687
-
Add autodiscover for kubernetes. 6055
-
Add Beats metrics reporting to Xpack. 3422
-
Update the command line library cobra and add support for zsh completion 5761
-
Update to Golang 1.9.2
-
Moved
ip_portindexer foradd_kubernetes_metadatato all beats. 5707 -
ip_portindexer now index both IP and IP:port pairs. 5721 -
Add the ability to write structured logs. 5901
-
Use structured logging for the metrics that are periodically logged via the
logging.metricsfeature. 5915 -
Improve Elasticsearch output metrics to count number of dropped and duplicate (if event ID is given) events. 5811
-
Add the abilility for the add_docker_metadata process to enrich based on process ID. 6100
-
The
add_docker_metadataandadd_kubernetes_metadataprocessors are now GA, instead of Beta. 6105 -
Update go-ucfg library to support top level key reference and cyclic key reference for the keystore 6098
Auditbeat
Filebeat
Metricbeat
-
Add ceph osd_df to metricbeat 5606
-
Add field network_names of hosts and virtual machines. 5646
-
Add experimental system/raid metricset. 5642
-
Add a dashboard for the Nginx module. 5991
-
Add experimental mongodb/collstats metricset. 5852
-
Update the MySQL dashboard to use the Time Series Visual Builder. 5996
-
Add experimental uwsgi module. 6006
-
Docker and Kubernetes modules are now GA, instead of Beta. 6105
-
Support haproxy stats gathering using http (additionaly to tcp socket). 5819
-
Support to optionally 'de dot' keys in http/json metricset to prevent collisions. 5957
Packetbeat
-
Configure good defaults for
add_kubernetes_metadata. 5707
Auditbeat
-
Add an error check to the file integrity scanner to prevent a panic when there is an error reading file info via lstat. 6005
Affecting all Beats
-
Remove ID() from Runner interface 5153
-
Correctly send configured
Hostheader to the remote server. 4842 -
Change add_kubernetes_metadata to attempt detection of namespace. 5482
-
Avoid double slash when join url and path 5517
-
Fix console color output for Windows. 5611
-
Fix logstash output debug message. https://github.com/elastic/beats/pull/5799{5799]
-
Fix isolation of modules when merging local and global field settings. 5795
-
Report ephemeral ID and uptime in monitoring events on all platforms 6501
Filebeat
Metricbeat
-
Change field type of http header from nested to object 5258
-
Fix the fetching of process information when some data is missing under MacOS X. 5337
-
Change
MySQL active connectionsvisualization title toMySQL total connections. 4812 -
Fix
ProcStateon Linux and FreeBSD when process names contain parentheses. 5775 -
Fix incorrect
Mem.Usedcalculation under linux. 5775 -
Fix
open_file_descriptor_countandmax_file_descriptor_countlost in zookeeper module 5902 -
Fix system process metricset for kernel processes. 5700
-
Change kubernetes.node.cpu.allocatable.cores to float. 6130
Packetbeat
-
Fix http status phrase parsing not allow spaces. 5312
-
Fix http parse to allow to parse get request with space in the URI. 5495
-
Fix mysql SQL parser to trim
\rfrom Windows ServerSELECT\r\n\t1. 5572 -
Fix corruption when parsing repeated headers in an HTTP request or response. 6325
-
Fix panic when parsing partial AMQP messages. 6384
-
Fix out of bounds access to slice in MongoDB parser. 6256
-
Fix sniffer hanging on exit under Linux. 6535
-
Fix bounds check error in http parser causing a panic. 6750
Winlogbeat
-
Fix the registry file. It was not correctly storing event log names, and upon restart it would begin reading at the start of each event log. 5813
-
Fix config validation to allow
event_logs.processors. [pull]6217[6217]
Affecting all Beats
-
Support dashboard loading without Elasticseach 5653
-
Changed the hashbang used in the beat helper script from
/bin/bashto/usr/bin/env bash. 5051 -
Changed beat helper script to use
execwhen running the beat. 5051 -
Fix reloader error message to only print on actual error 5066
-
Add support for enabling TLS renegotiation. 4386
-
Add Azure VM support for add_cloud_metadata processor 5355
-
Add
output.file.permissionconfig option. 4638 -
Refactor add_kubernetes_metadata to support autodiscovery 5434
-
Improve custom flag handling and CLI flags usage message. 5543
-
Add number_of_routing_shards config set to 30 5570
-
Set log level for kafka output. 5397
-
Move TCP UDP start up into
server.Start()4903 -
Update to Golang 1.9.2
Auditbeat
Filebeat
-
Add PostgreSQL module with slowlog support. 4763
-
Add Kafka log module. 4885
-
Add support for
/var/log/containers/log path inadd_kubernetes_metadataprocessor. 4981 -
Remove error log from runnerfactory as error is returned by API. 5085
-
Add experimental Docker
json-fileprospector . 5402 -
Add experimental Docker autodiscover functionality. 5245
-
Add option to convert the timestamps to UTC in the system module. 5647
-
Add Logstash module support for main log and the slow log, support the plain text or structured JSON format 5481
Metricbeat
-
Add graphite protocol metricbeat module. 4734
-
Add http server metricset to support push metrics via http. 4770
-
Make config object public for graphite and http server 4820
-
Add system uptime metricset. 4848
-
Add experimental
queuemetricset to RabbitMQ module. 4788 -
Add additional php-fpm pool status kpis for Metricbeat module 5287
-
Add etcd module. 4970
-
Add ip address of docker containers to event. 5379
-
Add ceph osd tree information to Metricbeat 5498
-
Add basic Logstash module. 5540
-
Add dashboard for Windows service metricset. 5603
-
Add experimental Docker autodiscover functionality. 5245
-
Add Windows service metricset in the windows module. 5332
-
Update gosigar to v0.6.0. 5775
Packetbeat
The list below covers the changes between 6.0.0-rc2 and 6.0.0 GA only.
Packetbeat
-
Remove not-working
runoptions.uidandrunoptions.gidoptions in Packetbeat. 5261
Affecting all Beats
Filebeat
Metricbeat
Metricbeat
-
Auto-select a hostname (based on the host on which the Beat is running) in the Host Overview dashboard. 5340
Filebeat
-
The
filebeat.config_diroption is deprecated. Usefilebeat.config.prospectoroptions instead. 5321
Affecting all Beats
-
Fix the
/usr/bin/beatnamescript to accept-d "*"as a parameter. 5040 -
Combine
fields.ymlproperties when they are defined in different sources. 5075 -
Keep Docker & Kubernetes pod metadata after container dies while they are needed by processors. 5084
-
Fix
fields.ymllookup when usingexport templatewith a custompath.configparam. 5089 -
Remove runner creation from every reload check 5141
-
Fix add_kubernetes_metadata matcher registry lookup. 5159
Metricbeat
-
Fix a memory allocation issue where more memory was allocated than needed in the windows-perfmon metricset. 5035
-
Don’t start metricbeat if external modules config is wrong and reload is disabled 5053
-
The MongoDB module now connects on each fetch, to avoid stopping the whole Metricbeat instance if MongoDB is not up when starting. 5120
-
Fix kubernetes events module to be able to index time fields properly. 5093
-
Fixed
cmd_setandcmd_getbeing mixed in the Memcache module. 5189
Affecting all Beats
-
The log directory (
path.log) for Windows services is now set toC:\ProgramData\[beatname]\logs. 4764 -
The _all field is disabled in Elasticsearch 6.0. This means that searching by individual words only work on text fields. 4901
-
Fail if removed setting output.X.flush_interval is explicitly configured.
-
Rename the
/usr/bin/beatname.shscript (e.g.metricbeat.sh) to/usr/bin/beatname. 4933 -
Beat does not start if elasticsearch index pattern was modified but not the template name and pattern. 4769
-
Fail if removed setting output.X.flush_interval is explicitly configured. 4880
Affecting all Beats
-
Register kubernetes
field_formatmatcher and remove logger inEncodeAPI 4888 -
Fix go plugins not loaded when beat starts 4799
-
Add support for
initContainersinadd_kubernetes_metadataprocessor. 4825 -
Eliminate deprecated default mapping in 6.x 4864
-
Fix pod name indexer to use both namespace, pod name to frame index key 4775
Filebeat
-
Fix issue where the
fileset.modulecould have the wrong value. 4761
Heartbeat
Metricbeat
Packetbeat
-
Update flow timestamp on each packet being received. 4895
Affecting all Beats
-
Add setting to enable/disable the slow start in logstash output. 4972
-
Update init scripts to use the
test configsubcommand instead of the deprecated-configtestflag. 4600 -
Get by default the credentials for connecting to Kibana from the Elasticsearch output configuration. 4867
-
Added
cloud.idandcloud.authsettings, for simplifying using Beats with the Elastic Cloud. 4959 -
Add lz4 compression support to kafka output. 4977
-
Add newer kafka versions to kafka output. 4977
-
Configure the index name when loading the dashboards and the index pattern. 4949
Metricbeat
Affecting all Beats
-
Rename
kubernetesprocessor toadd_kubernetes_metadata. 4473 -
Rename
.full.ymlconfig files to.reference.yml. 4563 -
The
scripts/import_dashboardsis removed from packages. Use thesetupcommand instead. 4586 -
Change format of the saved kibana dashboards to have a single JSON file for each dashboard 4413
-
Rename
configtestcommand totest config. 4590 -
Remove setting
queue_sizeandbulk_queue_size. 4650 -
Remove setting
dashboard.snapshotanddashboard.snapshot_url. They are no longer needed because the dashboards are included in the packages by default. 4675 -
Beats can no longer be launched from Windows Explorer (GUI), command line is required. 4420
Auditbeat
-
Changed file metricset config to make
file.pathsa list instead of a dictionary. 4796
Heartbeat
-
Renamed the heartbeat RPM/DEB name to
heartbeat-elastic. 4601
Metricbeat
-
Change all
system.cpu.*.pctmetrics to be scaled by the number of CPU cores. This will make the CPU usage percentages from the system cpu metricset consistent with the system process metricset. The documentation for these metrics already stated that on multi-core systems the percentages could be greater than 100%. 4544 -
Remove filters setting from metricbeat modules. 4699
-
Added
typefield to filesystem metrics. 4717
Packetbeat
-
Remove the already unsupported
pf_ringsniffer option. 4608
Affecting all Beats
Auditbeat
-
Fix
file.max_file_sizeconfig option for the audit file metricset. 4796
Filebeat
-
Fix issue where the
fileset.modulecould have the wrong value. 4761
Metricbeat
Packetbeat
-
Enabled /proc/net/tcp6 scanning and fixed ip v6 parsing. 4442
Winlogbeat
-
Removed validation of top-level config keys. This behavior was inconsistent with other Beats and caused maintainability issues. 4657
Affecting all Beats
-
New cli subcommands interface. 4420
-
Allow source path matching in
add_docker_metadataprocessor. 4495 -
Add support for analyzers and multifields in fields.yml. 4574
-
Add support for JSON logging. 4523
-
Add
test outputcommand, to test Elasticsearch and Logstash output settings. 4590 -
Introduce configurable event queue settings: queue.mem.events, queue.mem.flush.min_events and queue.mem.flush.timeout. 4650
-
Enable pipelining in Logstash output by default. 4650
-
Added 'result' field to Elasticsearch QueryResult struct for compatibility with 6.x Index and Delete API responses. {issue]4661[4661]
-
The sample dashboards are now included in the Beats packages. 4675
-
Add
patternoption to be used in the fields.yml to specify the pattern for a number field. 4731
Auditbeat
Filebeat
-
Add experimental Redis module. 4441
-
Nginx module: use the first not-private IP address as the remote_ip. 4417
-
Load Ingest Node pipelines when the Elasticsearch connection is established, instead of only once at startup. 4479
-
Add support for loading Xpack Machine Learning configurations from the modules, and added sample configurations for the Nginx module. 4506 4609
-
Add udp prospector type. 4452
-
Enabled Cgo which means libc is dynamically compiled. 4546
-
Add Beta module config reloading mechanism 4566
-
Remove spooler and publisher components and settings. 4644
Heartbeat
-
Enabled Cgo which means libc is dynamically compiled. 4546
Metricbeat
-
Add random startup delay to each metricset to avoid the thundering herd problem. 4010
-
Add the ability to configure audit rules to the kernel module. 4482
-
Add the ability to configure kernel’s audit failure mode. 4516
-
Add experimental Aerospike module. 4560
-
Vsphere module: collect custom fields from virtual machines. 4464
-
Add
test modulescommand, to test modules expected output. 4656 -
Add
processorssetting to metricbeat modules. 4699 -
Support
npipeprotocol (Windows) in Docker module. 4751
Winlogbeat
-
Add the ability to use LevelRaw if Level isn’t populated in the event XML. 4257
Auditbeat
-
Add file integrity metricset to the audit module. 4486
Filebeat
Affecting all Beats
Filebeat
Metricbeat
-
Set correct format for percent fields in memory module. 4619
-
Fix a debug statement that said a module wrapper had stopped when it hadn’t. 4264
-
Use MemAvailable value from /proc/meminfo on Linux 3.14. 4316
-
Fix panic when events were dropped by filters. 4327
-
Add filtering to system filesystem metricset to remove relative mountpoints like those from Linux network namespaces. 4370
-
Remove unnecessary print statement in schema apis. 4355
-
Fix type of field
haproxy.stat.check.health.last. 4407
Affecting all Beats
-
Upgraded to Golang 1.8.3. 4401
-
Added the possibility to set Elasticsearch mapping template settings from the Beat configuration file. 4284 4317
-
Add a variable to the SysV init scripts to make it easier to change the user. 4340
-
Add the option to write the generated Elasticsearch mapping template into a file. 4323
-
Add
instance_namein GCE add_cloud_metadata processor. 4414 -
Add
add_docker_metadataprocessor. 4352 -
Add
logging.filespermissionsoption. 4295
Filebeat - Added ability to sort harvested files. 4374 - Add experimental Redis slow log prospector type. 4180
Metricbeat
-
Add macOS implementation of the system diskio metricset. 4144
-
Add process_summary metricset that records high level metrics about processes. 4231
-
Add
kube-state-metricsbased metrics tokubernetesmodule 4253 -
Add debug logging to Jolokia JMX metricset. 4341
-
Add events metricset for kubernetes metricbeat module 4315
-
Change Metricbeat default configuration file to be better optimized for most users. 4329
-
Add experimental RabbitMQ module. 4394
-
Add Kibana dashboard for the Kubernetes modules. 4138
Packetbeat
Winlogbeat
Affecting all Beats
-
The
@metadata.typefield, added by the Logstash output, is deprecated, hardcoded todocand will be removed in future versions. 4331.
Filebeat
-
Deprecate
input_typeprospector config. Usetypeconfig option instead. 4294
-
If the Elasticsearch output is not enabled, but
setup.templateoptions are present (like it’s the case in the default Metricbeat configuration), the Beat stops with an error: "Template loading requested but the Elasticsearch output is not configured/enabled". To avoid this error, disable the template loading explicitlysetup.template.enabled: false.
Affecting all Beats
-
Introduce beat version in the Elasticsearch index and mapping template 3527
-
Usage of field
_typeis now ignored and hardcoded todoc. 3757 -
Change vendor manager from glide to govendor. 3851
-
Rename
errorfield toerror.message. 3987 -
Change
dashboards.config options tosetup.dashboards.. 3921 -
Change
outputs.elasticsearch.template.* to `setup.template.*4080
Filebeat
-
Remove code to convert states from 1.x. 3767
-
Remove deprecated config options
force_close_filesandclose_older. 3768 -
Change
clean_removedbehaviour to also remove states for files which cannot be found anymore under the same name. 3827 -
Remove
document_typeconfig option. Usefieldsinstead. 4204 -
Move
json_errorundererror.messageanderror.key. 4167
Packetbeat
Winlogbeat
-
Remove metrics endpoint. Replaced by http endpoint in libbeat (see #3717). 3901
Affecting all Beats
-
Add
_id,_type,_indexand_scorefields in the generated index pattern. 3282
Filebeat
Heartbeat
-
Use IP type of elasticsearch for ip field. 3926
Metricbeat
Affecting all Beats
-
Initialize a beats UUID from file on startup. 3615
-
Add new
add_localeprocessor to export the local timezone with an event. 3902 -
Add http endpoint. 3717
-
Updated to Go 1.8.1. 4033
-
Add kubernetes processor 3888
-
Add support for
include_labelsandinclude_annotationsin kubernetes processor 4043 -
Support new
index_patternsfield when loading templates for Elasticsearch >= 6.0 4056 -
Adding goimports support to make check and fmt 4114
-
Make kubernetes indexers/matchers pluggable 4151
-
Abstracting pod interface in kubernetes plugin to enable easier vendoring 4152
Filebeat
-
Restructure
input.Eventto be inline withoutputs.Data3823 -
Add base for supporting prospector level processors 3853
-
Add
filebeat.config.pathas replacement forconfig_dir. 4051 -
Add a
recursive_glob.enabledsetting to expand**in patterns. 3980 -
Add Icinga module. 3904
-
Add ability to parse nginx logs exposing the X-Forwarded-For header instead of the remote address.
Heartbeat
-
Event format and field naming changes in Heartbeat and sample Dashboard. 4091
Metricbeat
-
Add experimental metricset
perfmonto Windows module. 3758 -
Add memcached module with stats metricset. 3693
-
Add the
process.cmdline.cache.enabledconfig option to the System Process Metricset. 3891 -
Add new MetricSet interfaces for developers (
Closer,ReportingFetcher, andPushMetricSet). 3908 -
Add kubelet module 3916
-
Add dropwizard module 4022
-
Adding query APIs for metricsets and modules from metricbeat registry 4102
-
Fixing nil pointer on prometheus collector when http response is nil 4119
-
Add http module with json metricset. 4092
-
Add the option to the system module to include only the first top N processes by CPU and memory. 4127.
-
Add experimental Vsphere module. 4028
-
Add experimental Elasticsearch module. 3903
-
Add experimental Kibana module. 3895
-
Move elasticsearch metricset node_stats under node.stats namespace. 4142
-
Make IP port indexer constructor public 4434
Packetbeat
Winlogbeat
Affecting all Beats
-
The _all.norms setting in the Elasticsearch template is no longer disabled. This increases the storage size with one byte per document, but allows for a better upgrade experience to 6.0. 4901
Filebeat
-
Fix issue where the
fileset.modulecould have the wrong value. 4761
Packetbeat
-
Update flow timestamp on each packet being received. 4895
Metricbeat
Affecting all Beats
-
Add option to the import_dashboards script to load the dashboards via Kibana API. 4682
Filebeat
Metricbeat
-
Add
filesystem.ignore_typesto system module for ignoring filesystem types. 4685
Affecting all Beats
-
Loading more than one output is deprecated and will be removed in 6.0. 4907
Affecting all Beats
-
Usage of field
_typeis now ignored and hardcoded todoc. 3757
Metricbeat
- Change all system.cpu.*.pct metrics to be scaled by the number of CPU cores.
This will make the CPU usage percentages from the system cpu metricset consistent
with the system process metricset. The documentation for these metrics already
stated that on multi-core systems the percentages could be greater than 100%. 4544
Affecting all Beats
Filebeat
Affecting all Beats
Filebeat
Metricbeat
Winlogbeat
-
Fix null terminators include in raw XML string when include_xml is enabled. 3943
Affecting all Beats
-
Update index mappings to support future Elasticsearch 6.X. 3778
Filebeat
Heartbeat
-
Add default ports in HTTP monitor. 3924
Metricbeat
-
Add beta Jolokia module. 3844
-
Add dashboard for the MySQL module. 3716
-
Module configuration reloading is now beta instead of experimental. 3841
-
Marked http fields from the HAProxy module optional to improve compatibility with 1.5. 3788
-
Add support for custom HTTP headers and TLS for the Metricbeat modules. 3945
Packetbeat
Affecting all Beats
Filebeat
Metricbeat
-
Avoid errors when some Apache status fields are missing. 3074
Affecting all Beats
Filebeat
-
Always use absolute path for event and registry. This can lead to issues when relative paths were used before. 3328
Metricbeat
Affecting all Beats
-
Add
_id,_type,_indexand_scorefields in the generated index pattern. 3282
Filebeat - Always use absolute path for event and registry. 3328 - Raise an exception in case there is a syntax error in one of the configuration files available under filebeat.config_dir. 3573 - Fix empty registry file on machine crash. 3537
Metricbeat
Winlogbeat
-
Fix handling of empty strings in event_data. 3705
Affecting all Beats
-
Files created by Beats (logs, registry, file output) will have 0600 permissions. 3387.
-
RPM/deb packages will now install the config file with 0600 permissions. 3382
-
Add the option to pass custom HTTP headers to the Elasticsearch output. 3400
-
Unify
regexpandcontainsconditionals, for both to support array of strings and convert numbers to strings if required. 3469 -
Add the option to load the sample dashboards during the Beat startup phase. 3506
-
Disabled date detection in Elasticsearch index templates. Date fields must be explicitly defined in index templates. 3528
-
Using environment variables in the configuration file is now GA, instead of experimental. 3525
Filebeat
-
Add Filebeat modules for system, apache2, mysql, and nginx. 3159
-
Add the
pipelineconfig option at the prospector level, for configuring the Ingest Node pipeline ID. 3433 -
Update regular expressions used for matching file names or lines (multiline, include/exclude functionality) to new matchers improving performance of simple string matches. 3469
-
The
symlinksandharverster_limitsettings are now GA, instead of experimental. 3525 -
close_timeout is also applied when the output is blocking. 3511
-
Improve handling of different path variants on Windows. 3781
-
Add multiline.flush_pattern option, for specifying the 'end' of a multiline pattern 4019
Heartbeat
-
Add
tags,fieldsandfields_under_rootin monitors configuration. 3623
Metricbeat
-
Add experimental dbstats metricset to MongoDB module. 3228
-
Use persistent, direct connections to the configured nodes for MongoDB module. 3228
-
Add dynamic configuration reloading for modules. 3281
-
Add docker health metricset 3357
-
Add docker image metricset 3467
-
System module uses new matchers for white-listing processes. 3469
-
Add Beta CEPH module with health metricset. 3311
-
Add Beta php_fpm module with pool metricset. 3415
-
The Docker, Kafka, and Prometheus modules are now Beta, instead of experimental. 3525
-
The HAProxy module is now GA, instead of experimental. 3525
-
Add the ability to collect the environment variables from system processes. 3337
Metricbeat
Affecting all Beats
-
Fix overwriting explicit empty config sections. 2918
Filebeat
-
Fix alignment issue were Filebeat compiled with Go 1.7.4 was crashing on 32 bits system. 3273
Metricbeat
Packetbeat
-
Fix issue where some Cassandra visualizations were showing data from all protocols. 3314
Affecting all Beats
-
Add support for passing list and dictionary settings via -E flag.
-
Support for parsing list and dictionary setting from environment variables.
-
Added new flags to import_dashboards (-cacert, -cert, -key, -insecure). 3139 3163
-
The limit for the number of fields is increased via the mapping template. 3275
-
Updated to Go 1.7.4. 3277
-
Added a NOTICE file containing the notices and licenses of the dependencies. 3334.
Heartbeat
-
First release, containing monitors for ICMP, TCP, and HTTP.
Filebeat
Metricbeat
-
Kafka module broker matching enhancements. 3129
-
Add a couchbase module with metricsets for node, cluster and bucket. 3081
-
Export number of cores for CPU module. 3192
-
Experimental Prometheus module. 3202
-
Add system socket module that reports all TCP sockets. 3246
-
Kafka consumer groups metricset. 3240
-
Add jolokia module with dynamic jmx metricset. 3570
Winlogbeat
Metricbeat
-
Change data structure of experimental haproxy module. 3003
Filebeat
-
If a file is falling under
ignore_olderduring startup, offset is now set to end of file instead of 0. With the previous logic the whole file was sent in case a line was added and it was inconsistent with files which were harvested previously. 2907 -
tail_filesis now only applied on the first scan and not for all new files. 2932
Affecting all Beats
-
Fix empty benign errors logged by processor actions. 3046
Metricbeat
-
Calculate the fsstat values per mounting point, and not filesystem. 2777
Affecting all Beats
Metricbeat
-
Add experimental Docker module. Provided by Ingensi and @douaejeouit based on dockbeat.
-
Add a sample Redis Kibana dashboard. 2916
-
Add support for MongoDB 3.4 and WiredTiger metrics. 2999
-
Add experimental kafka module with partition metricset. 2969
-
Add raw config option for mysql/status metricset. 3001
-
Add command fields for mysql/status metricset. 3251
Filebeat
-
Add command line option
-onceto run Filebeat only once and then close. 2456 -
Only load matching states into prospector to improve state handling 2840
-
Reset all states ttl on startup to make sure it is overwritten by new config 2840
-
Persist all states for files which fall under
ignore_olderto have consistent behaviour 2859 -
Improve shutdown behaviour with large number of files. 3035
Winlogbeat
-
Add
event_logs.batch_read_sizeconfiguration option. 2641
Version 5.1.0 doesn’t exist because, for a short period of time, the Elastic Yum and Apt repositories included unreleased binaries labeled 5.1.0. To avoid confusion and upgrade issues for the people that have installed these without realizing, we decided to skip the 5.1.0 version and release 5.1.1 instead.
Metricbeat
-
Fix
system.process.start_timeon Windows. 2848 -
Fix
system.process.ppidon Windows. 2860 -
Fix system process metricset for Windows XP and 2003.
cmdlinewill be unavailable. 1704 -
Fix access denied issues in system process metricset by enabling SeDebugPrivilege on Windows. 1897
-
Fix system diskio metricset for Windows XP and 2003. 2885
Packetbeat
-
Fix 'index out of bounds' bug in Packetbeat DNS protocol plugin. 2872
Filebeat
-
Fix registry cleanup issue when files falling under ignore_older after restart. 2818
The list below covers the changes between 5.0.0-rc1 and 5.0.0 GA only.
Affecting all Beats
Metricbeat
-
Fix high CPU usage on macOS when encountering processes with long command lines. 2747
-
Fix high value of
system.memory.actual.freeandsystem.memory.actual.used. 2653 -
Change several
OpenProcesscalls on Windows to request the lowest possible access provilege. 1897 -
Fix system.memory.actual.free high value on Windows. 2653
Filebeat
-
Fix issue when clean_removed and clean_inactive were used together that states were not directly removed from the registry.
-
Fix issue where upgrading a 1.x registry file resulted in duplicate state entries. 2792
Affecting all Beats
-
A dynamic mapping rule is added to the default Elasticsearch template to treat strings as keywords by default. 2688
Affecting all Beats
-
Make sure Beats sent always float values when they are defined as float by sending 5.00000 instead of 5. 2627
-
Fix ignoring all fields from drop_fields in case the first field is unknown. 2685
-
Fix dynamic configuration int/uint to float type conversion. 2698
-
Fix primitive types conversion if values are read from environment variables. 2698
Metricbeat
-
Fix default configuration file on Windows to not enabled the
loadmetricset. 2632
Packetbeat
-
Fix the
bpf_filtersetting. 2660
Filebeat
-
Fix input buffer on encoding problem. 2416
Affecting all Beats
-
Change Elasticsearch output index configuration to be based on format strings. If index has been configured, no date will be appended anymore to the index name. 2119
-
Replace
output.kafka.use_typebyoutput.kafka.topicaccepting a format string. 2188 -
If the path specified by the
-cflag is not absolute and-path.configis not specified, it is considered relative to the current working directory. 2245 -
rename
tlsconfigurations section tossl. 2330 -
rename
certificate_keyconfiguration tokey. 2330 -
replace
tls.insecurewithssl.verification_modesetting. 2330 -
replace
tls.min/max_versionwithssl.supported_protocolssetting requiring full protocol name. 2330
Metricbeat
-
Change field type system.process.cpu.start_time from keyword to date. 1565
-
redis/info metricset fields were renamed up according to the naming conventions.
Packetbeat
Filebeat
-
Set close_inactive default to 5 minutes (was 1 hour before)
-
Set clean_removed and close_removed to true by default
Affecting all Beats
-
Fix logstash output handles error twice when asynchronous sending fails. 2441
-
Fix Elasticsearch structured error response parsing error. 2229
-
Fixed the run script to allow the overriding of the configuration file. 2171
-
Fix logstash output crash if no hosts are configured. 2325
-
Fix array value support in -E CLI flag. 2521
-
Fix merging array values if -c CLI flag is used multiple times. 2521
-
Fix beats failing to start due to invalid duplicate key error in configuration file. 2521
-
Fix panic on non writable logging directory. 2571
Metricbeat
-
Fix module filters to work properly with drop_event filter. 2249
Packetbeat
Filebeat
Winlogbeat
-
Fix corrupt registry file that occurs on power loss by disabling file write caching. 2313
Affecting all Beats
-
Add script to generate the Kibana index-pattern from fields.yml. 2122
-
Enhance Redis output key selection based on format string. 2169
-
Configurable Redis
keysusing filters and format strings. 2169 -
Add format string support to
output.kafka.topic. 2188 -
Add
output.kafka.topicsfor more advanced kafka topic selection per event. 2188 -
Add support for Kafka 0.10. 2190
-
Add SASL/PLAIN authentication support to kafka output. 2190
-
Make Kafka metadata update configurable. 2190
-
Add Kafka version setting (optional) enabling kafka broker version support. 2190
-
Add Kafka message timestamp if at least version 0.10 is configured. 2190
-
Add configurable Kafka event key setting. 2284
-
Add settings for configuring the kafka partitioning strategy. 2284
-
Add partitioner settings
reachable_onlyto ignore partitions not reachable by network. 2284 -
Enhance contains condition to work on fields that are arrays of strings. 2237
-
Lookup the configuration file relative to the
-path.configCLI flag. 2245 -
Re-write import_dashboards.sh in Golang. 2155
-
Update to Go 1.7. 2306
-
Log total non-zero internal metrics on shutdown. 2349
-
Add support for encrypted private key files by introducing
ssl.key_passphrasesetting. 2330 -
Add experimental symlink support with
symlinksconfig 2478 -
Improve validation of registry file on startup.
Metricbeat
-
Use the new scaled_float Elasticsearch type for the percentage values. 2156
-
Add experimental cgroup metrics to the system/process MetricSet. 2184
-
Added a PostgreSQL module. 2253
-
Improve mapping by converting half_float to scaled_float and integers to long. 2430
-
Add experimental haproxy module. 2384
-
Add Kibana dashboard for cgroups data 2555
Packetbeat
Filebeat
Affecting all Beats
-
Rename the
filterssection toprocessors. 1944 -
Introduce the condition with
whenin the processor configuration. 1949 -
The Elasticsearch template is now loaded by default. 1993
-
The Redis output
indexsetting is renamed tokey.indexstill works but it’s deprecated. 2077 -
The undocumented file output
indexsetting was removed. Usefilenameinstead. 2077
Metricbeat
Packetbeat
-
Set
enabled` inpacketbeat.protocols.icmpconfiguration totrueby default. 1988
Affecting all Beats
-
Fix sync publisher
PublishEventsreturn value if client is closed concurrently. 2046
Metricbeat
-
Do not send zero values when no value was present in the source. 1972
Filebeat
Winlogbeat
-
Fix potential data loss between Winlogbeat restarts, reporting unpublished lines as published. 2041
Affecting all Beats
-
Periodically log internal metrics. 1955
-
Add enabled setting to all output modules. 1987
-
Command line flag
-ccan be used multiple times. 1985 -
Add OR/AND/NOT to the condition associated with the processors. 1983
-
Add
-ECLI flag for overwriting single config options via command line. 1986 -
Choose the mapping template file based on the Elasticsearch version. 1993
-
Check stdout being available when console output is configured. 2035
Metricbeat
-
Add pgid field to process information. https://github.com/elastic/beats/pull/ 2021[2021]
Packetbeat
Filebeat
Affecting all Beats
-
The topology_expire option of the Elasticserach output was removed. 1907
Filebeat
-
Stop following symlink. Symlinks are now ignored: 1686
Affecting all Beats
-
Reset backoff factor on partial ACK. 1803
-
Fix beats load balancer deadlock if max_retries: -1 or publish_async is enabled in filebeat. 1829
-
Fix logstash output with pipelining mode enabled not reconnecting. 1876
-
Empty configuration sections become merge-able with variables containing full path. 1900
-
Fix error message about required fields missing not printing the missing field name. 1900
Metricbeat
-
Fix the CPU values returned for each core. 1863
Packetbeat
Winlogbeat
-
Fix issue with rendering forwarded event log records. 1891
Affecting all Beats
-
All configuration settings under
shipper:are moved to be top level configuration settings. I.e.shipper.name:becomesname:in the configuration file. 1570
Topbeat
-
Topbeat is replaced by Metricbeat.
Filebeat
-
The state for files which fall under ignore_older is not stored anymore. This has the consequence, that if a file which fell under ignore_older is updated, the whole file will be crawled.
Affecting all Beats
-
Add conditions to generic filtering. 1623
Metricbeat
-
First public release, containing the following modules: apache, mysql, nginx, redis, system, and zookeeper.
Filebeat
-
The registry format was changed to an array instead of dict. The migration to the new format will happen automatically at the first startup. 1703
Affecting all Beats
-
The support for doing GeoIP lookups is deprecated and will be removed in version 6.0. 1601
Affecting all Beats
Packetbeat
Filebeat
-
Default location for the registry file was changed to be
data/registryfrom the binary directory, rather than.filebeatin the current working directory. This affects installations for zip/tar.gz/source, the location for DEB and RPM packages stays the same. 1373
Affecting all Beats
-
Drain response buffers when pipelining is used by Redis output. 1353
-
Unterminated environment variable expressions in config files will now cause an error 1389
-
Fix issue with the automatic template loading when Elasticsearch is not available on Beat start. 1321
-
Fix bug affecting -cpuprofile, -memprofile, and -httpprof CLI flags 1415
-
Fix race when multiple outputs access the same event with logstash output manipulating event 1410 1428
-
Seed random number generator using crypto.rand package. https://github.com/elastic/beats/pull/1503{1503]
-
Fix beats hanging in -configtest 1213
-
Fix kafka log message output 1516
Filebeat
-
Improvements in registrar dealing with file rotation. 1281
-
Fix issue with JSON decoding where
@timestamportypekeys with the wrong type could cause Filebeat to crash. 1378 -
Fix issue with JSON decoding where values having
nullas values could crash Filebeat. 1466 -
Multiline reader normalizing newline to use
\n. 1552
Winlogbeat
Affecting all Beats
-
Add support for TLS to Redis output. 1353
-
Add SOCKS5 proxy support to Redis output. 1353
-
Failover and load balancing support in redis output. 1353
-
Multiple-worker per host support for redis output. 1353
-
Added ability to escape
${x}in config files to avoid environment variable expansion 1389 -
Configuration options and CLI flags for setting the home, data and config paths. 1373
-
Configuration options and CLI flags for setting the default logs path. 1437
-
Update to Go 1.6.2 1447
-
Add Elasticsearch template files compatible with Elasticsearch 2.x. 1501
-
Add scripts for managing the dashboards of a single Beat 1359
Packetbeat
-
Fix compile issues for OpenBSD. 1347
Topbeat
-
Updated elastic/gosigar version so Topbeat can compile on OpenBSD. 1403
libbeat
Packetbeat
-
Rename output fields in the dns package. Former flag
recursion_allowedbecomesrecursion_available. 803 Former SOA fieldttlbecomesminimum. 803 -
The fully qualified domain names which are part of output fields values of the dns package now terminate with a dot. 803
-
Remove the count field from the exported event 1210
Topbeat
Filebeat
Winlogbeat
Affecting all Beats
-
Logstash output will not retry events that are not JSON-encodable 927
Packetbeat
Topbeat
-
Fix issue with
cpu.system_pbeing greater than 1 on Windows 1128
Filebeat
Winlogbeat
Affecting all Beats
-
Update builds to Golang version 1.6
-
Add option to Elasticsearch output to pass http parameters in index operations 805
-
Improve Logstash and Elasticsearch backoff behavior. 927
-
Add experimental Kafka output. 942
-
Add config file option to configure GOMAXPROCS. 969
-
Improve shutdown handling in libbeat. 1075
-
Add
fieldsandfields_under_rootoptions under theshipperconfiguration 1092 -
Add the ability to use a SOCKS5 proxy with the Logstash output 823
-
The
-configtestflag will now print "Config OK" to stdout on success 1249
Packetbeat
Topbeat
-
Add
usernameto processes 845
Filebeat
Winlogbeat
-
Add caching of event metadata handles and the system render context for the wineventlog API 888
-
Improve config validation by checking for unknown top-level YAML keys. 1100
-
Add the ability to set tags, fields, and fields_under_root as options for each event log 1092
-
Add additional data to the events published by Winlogbeat. The new fields are
activity_id,event_data,keywords,opcode,process_id,provider_guid,related_activity_id,task,thread_id,user_data, andversion. 1053 -
Add
event_id,level, andproviderconfiguration options for filtering events 1218 -
Add
include_xmlconfiguration option for including the raw XML with the event 1218
-
All Beats can hang or panic on shutdown if the next server in the pipeline (e.g. Elasticsearch or Logstash) is not reachable. 1319
-
When running the Beats as a service on Windows, you need to manually load the Elasticsearch mapping template. 1315
-
The ES template automatic load doesn’t work if Elasticsearch is not available when the Beat is starting. 1321
Filebeat
-
Undocumented support for following symlinks is deprecated. Filebeat will not follow symlinks in version 5.0. 1767
Affecting all Beats
Packetbeat
Affecting all Beats
Topbeat
Filebeat
-
Default config for ignore_older is now infinite instead of 24h, means ignore_older is disabled by default. Use close_older to only close file handlers.
Packetbeat
-
Split real_ip_header value when it contains multiple IPs 1241
Winlogbeat
-
Fix invalid
event_idon Windows XP and Windows 2003 1227
Affecting all Beats
Packetbeat
-
Fix setting direction to out and use its value to decide when dropping events if ignore_outgoing is enabled 557
-
Fix logging issue with file-based output where newlines could be misplaced during concurrent logging 650
-
Reduce memory usage by having separate queue sizes for single events and bulk events. 649 516
-
Set default bulk_max_size value to 2048 628
-
Fix logstash window size of 1 not increasing. 598
Packetbeat
Filebeat
-
Set spool_size default value to 2048 628
Affecting all Beats
Packetbeat
Topbeat
-
Group all CPU usage per core statistics and export them optionally if cpu_per_core is configured 496
Filebeat
Winlogbeat
-
First public release of Winlogbeat
Filebeat
-
Fix force_close_files in case renamed file appeared very fast. 302
Packetbeat
-
Improve MongoDB message correlation. 377
-
Improve redis parser performance. 422
-
Fix panic on nil in redis protocol parser. 384
-
Fix errors redis parser when messages are split in multiple TCP segments. 402
-
Fix errors in redis parser when length prefixed strings contain sequences of CRLF. 402
-
Fix errors in redis parser when dealing with nested arrays. 402
Affecting all Beats
-
Fix random panic on shutdown by calling shutdown handler only once. elastic/filebeat#204
-
Fix credentials are not send when pinging an elasticsearch host. elastic/fileabeat#287
Filebeat
-
Fix problem that harvesters stopped reading after some time and filebeat stopped processing events #257
-
Fix line truncating by internal buffers being reused by accident #258
-
Set default ignore_older to 24 hours #282
Affecting all Beats
-
The
shipperoutput field is renamed tobeat.name. #285 -
Use of
enabledas a configuration option for outputs (elasticsearch, logstash, etc.) has been removed. #264 -
Use of
disabledas a configuration option for tls has been removed. #264 -
The
-testcommand line flag was renamed to-configtest. #264 -
Disable geoip by default. To enable it uncomment in config file. #305
Filebeat
-
Removed utf-16be-bom encoding support. Support will be added with fix for #205
-
Rename force_close_windows_files to force_close_files and make it available for all platforms.
Affecting all Beats
-
Disable logging to stderr after configuration phase. #276
-
Set the default file logging path when not set in config. #275
-
Fix bug silently dropping records based on current window size. elastic/filebeat#226
-
Fix direction field in published events. #300
-
Fix elasticsearch structured errors breaking error handling. #309
Packetbeat
-
Packetbeat will now exit if a configuration error is detected. #357
-
Fixed an issue handling DNS requests containing no questions. #369
Topbeat
-
Fix leak of Windows handles. #98
-
Fix memory leak of process information. #104
Filebeat
-
Filebeat will now exit if a configuration error is detected. #198
-
Fix to enable prospector to harvest existing files that are modified. #199
-
Improve line reading and encoding to better keep track of file offsets based on encoding. #224
-
Set input_type by default to "log"
Affecting all Beats
-
Rename timestamp field with @timestamp. #237
Packetbeat
-
Rename timestamp field with @timestamp. #343
Topbeat
-
Rename timestamp field with @timestamp for a better integration with Logstash. #80
Filebeat
-
Rename the timestamp field with @timestamp #168
-
Rename tail_on_rotate prospector config to tail_files
-
Removal of line field in event. Line number was not correct and does not add value. #217
Affecting all Beats
-
Use stderr for console log output. #219
-
Handle empty event array in publisher. #207
-
Respect '*' debug selector in IsDebug. #226 (elastic#339)
-
Limit number of workers for Elasticsearch output. elastic#226
-
On Windows, remove service related error message when running in the console. #242
-
Fix waitRetry no configured in single output mode configuration. elastic/filebeat#144
-
Use http as the default scheme in the elasticsearch hosts #253
-
Respect max bulk size if bulk publisher (collector) is disabled or sync flag is set.
-
Always evaluate status code from Elasticsearch responses when indexing events. #192
-
Use bulk_max_size configuration option instead of bulk_size. #256
-
Fix max_retries=0 (no retries) configuration option. #266
-
Filename used for file based logging now defaults to beat name. #267
Packetbeat
-
Close file descriptors used to monitor processes. #337
-
Remove old RPM spec file. It moved to elastic/beats-packer. #334
Topbeat
-
Don’t wait for one period until shutdown #75
Filebeat
-
Omit 'fields' from event JSON when null. #126
-
Make offset and line value of type long in elasticsearch template to prevent overflow. #140
-
Fix locking files for writing behaviour. #156
-
Introduce 'document_type' config option per prospector to define document type for event stored in elasticsearch. #133
-
Add 'input_type' field to published events reporting the prospector type being used. #133
-
Fix high CPU usage when not connected to Elasticsearch or Logstash. #144
-
Fix issue that files were not crawled anymore when encoding was set to something other then plain. #182
Affecting all Beats
-
Add Console output plugin. #218
-
Add timestamp to log messages #245
-
Send @metadata.beat to Logstash instead of @metadata.index to prevent possible name clashes and give user full control over index name used for Elasticsearch
-
Add logging messages for bulk publishing in case of error #229
-
Add option to configure number of parallel workers publishing to Elasticsearch or Logstash.
-
Set default bulk size for Elasticsearch output to 50.
-
Set default http timeout for Elasticsearch to 90s.
-
Improve publish retry if sync flag is set by retrying only up to max bulk size events instead of all events to be published.
Filebeat
-
Introduction of backoff, backoff_factor, max_backoff, partial_line_waiting, force_close_windows_files config variables to make crawling more configurable.
-
All Godeps dependencies were updated to master on 2015-10-21 [#122]
-
Set default value for ignore_older config to 10 minutes. #164
-
Added the fields_under_root setting to optionally store the custom fields top level in the output dictionary. #188
-
Add more encodings by using x/text/encodings/htmlindex package to select encoding by name.
Affecting all Beats
-
Update tls config options naming from dash to underline #162
-
Feature/output modes: Introduction of PublishEvent(s) to be used by beats #118 #115
Packetbeat
-
Renamed http module config file option 'strip_authorization' to 'redact_authorization'
-
Save_topology is set to false by default
-
Rename elasticsearch index to [packetbeat-]YYYY.MM.DD
Topbeat
-
Percentage fields (e.g user_p) are exported as a float between 0 and 1 #34
Affecting all Beats
-
Determine Elasticsearch index for an event based on UTC time #81
-
Fixing ES output’s defaultDeadTimeout so that it is 60 seconds #103
-
ES outputer: fix timestamp conversion #91
-
Fix TLS insecure config option #239
-
ES outputer: check bulk API per item status code for retransmit on failure.
Packetbeat
-
Support for lower-case header names when redacting http authorization headers
-
Redact proxy-authorization if redact-authorization is set
-
Fix some multithreading issues #203
-
Fix negative response time #216
-
Fix memcache TCP connection being nil after dropping stream data. #299
-
Add missing DNS protocol configuration to documentation #269
Topbeat
-
Don’t divide the reported memory by an extra 1024 #60
Affecting all Beats
-
Add logstash output plugin #151
-
Integration tests for Beat → Logstash → Elasticsearch added #195 #188 #168 #137 #128 #112
-
Large updates and improvements to the documentation
-
Add direction field to publisher output to indicate inbound/outbound transactions #150
-
Add tls configuration support to elasticsearch and logstash outputers #139
-
All external dependencies were updated to the latest version. Update to Golang 1.5.1 #162
-
Guarantee ES index is based in UTC time zone #164
-
Cache: optional per element timeout #144
-
Make it possible to set hosts in different ways. #135
-
Expose more TLS config options #124
-
Use the Beat name in the default configuration file path #99
Packetbeat
-
add [.editorconfig file](http://editorconfig.org/)
-
add (experimental/unsupported?) saltstack files
-
Sample config file cleanup
-
Moved common documentation to [libbeat repository](https://github.com/elastic/libbeat)
-
Update build to go 1.5.1
-
Adding device descriptions to the -device output.
-
Generate coverage for system tests
-
Move go-daemon dependency to beats-packer
-
Rename integration tests to system tests
-
Made the
-devicesoption more user friendly in casesudois not used. Issue #296. -
Publish expired DNS transactions #301
-
Update protocol guide to libbeat changes
-
Add protocol registration to new protocol guide
-
Make transaction timeouts configurable #300
-
Add direction field to the exported fields #317
Topbeat
-
Document fields in a standardized format (etc/fields.yml) #34
-
Updated to use new libbeat Publisher #37 #41
-
Update to go 1.5.1 #43
-
Updated configuration files with comments for all options #65
-
Documentation improvements