You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It took me hours to debug this issue, and it wasn't until digging into the plugin's README files that I figured out what is going on!
Both the Opensearch & Elasticsearch plugins mention the logstash_format parameter impacting the index. What it does not say is that if you set logstash_format = false, it also sets include_timestamp=false. This results in records not being usable in Opensearch Dashboards & Kibana, as there are no time records.
If true, Fluentd uses the conventional index name format logstash-%Y.%m.%d (default: false). This option supersedes the index_name option.
In addition, include_timestamp (default: false) is set to true, which sends timestamp information which can be used by (Opensearch Dashboards / Kibana).
Describe the bug
It took me hours to debug this issue, and it wasn't until digging into the plugin's README files that I figured out what is going on!
Both the Opensearch & Elasticsearch plugins mention the logstash_format parameter impacting the index. What it does not say is that if you set logstash_format = false, it also sets include_timestamp=false. This results in records not being usable in Opensearch Dashboards & Kibana, as there are no time records.
Link to the problematic documentation
https://docs.fluentd.org/output/opensearch#logstash_format-optional
https://docs.fluentd.org/output/elasticsearch#logstash_format-optional
Expected explanation
If true, Fluentd uses the conventional index name format logstash-%Y.%m.%d (default: false). This option supersedes the
index_name
option.In addition,
include_timestamp
(default: false) is set to true, which sends timestamp information which can be used by (Opensearch Dashboards / Kibana).Additional context
The documentation here: https://github.com/fluent/fluent-plugin-opensearch#include_timestamp does indicate this behavior.
Note that the documentation here is much more thorough and authoritative overall.
The text was updated successfully, but these errors were encountered: