From 5606e5e30d516a000d01d38c86b5cd14f1af2bce Mon Sep 17 00:00:00 2001 From: Lynette Miles Date: Mon, 2 Mar 2026 16:45:17 -0800 Subject: [PATCH 1/4] Bad link report: fixing broken links and redirected URLs Signed-off-by: Lynette Miles --- CONTRIBUTING.md | 2 +- about/fluentd-and-fluent-bit.md | 2 +- administration/monitoring.md | 2 +- installation/downloads.md | 2 +- pipeline/filters/aws-metadata.md | 2 +- pipeline/filters/kubernetes.md | 2 +- pipeline/inputs/cpu-metrics.md | 2 +- pipeline/inputs/kafka.md | 2 +- .../inputs/windows-event-log-winevtlog.md | 2 +- pipeline/outputs/bigquery.md | 20 +-- pipeline/outputs/chronicle.md | 8 +- pipeline/outputs/cloudwatch.md | 2 +- pipeline/outputs/datadog.md | 8 +- pipeline/outputs/dynatrace.md | 7 +- pipeline/outputs/elasticsearch.md | 7 +- pipeline/outputs/firehose.md | 4 +- pipeline/outputs/gelf.md | 2 +- pipeline/outputs/influxdb.md | 2 +- pipeline/outputs/kafka.md | 4 +- pipeline/outputs/loki.md | 2 +- pipeline/outputs/observe.md | 4 +- pipeline/outputs/openobserve.md | 4 +- pipeline/outputs/parseable.md | 2 +- pipeline/outputs/prometheus-remote-write.md | 6 +- pipeline/outputs/s3.md | 156 +++++++++--------- pipeline/outputs/slack.md | 8 +- pipeline/outputs/stackdriver.md | 18 +- 27 files changed, 141 insertions(+), 141 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8949ac40d..3612c685e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -127,7 +127,7 @@ This repository runs linters as GitHub Actions for each pull request. If a linte ### Vale -[Vale](https://vale.sh/docs/) lints prose for style and clarity. In addition to reviewing the results of each Vale test in GitHub, you can use the [Vale plugin for VSCode](https://marketplace.visualstudio.com/items?itemName=ChrisChinchilla.vale-vscode) to view errors and suggestions locally. +[Vale](https://vale.sh/docs) lints prose for style and clarity. In addition to reviewing the results of each Vale test in GitHub, you can use the [Vale plugin for VSCode](https://marketplace.visualstudio.com/items?itemName=ChrisChinchilla.vale-vscode) to view errors and suggestions locally. Vale tests for the Fluent Bit docs are stored in the [`/vale-styles`](https://github.com/fluent/fluent-bit-docs/tree/master/vale-styles) folder. Most Vale tests are at the `suggestion` or `warning` level and won't block pull requests from merging. However, tests at the `error` level will block merging until the associated issue is fixed. diff --git a/about/fluentd-and-fluent-bit.md b/about/fluentd-and-fluent-bit.md index f81acef69..05459a5e1 100644 --- a/about/fluentd-and-fluent-bit.md +++ b/about/fluentd-and-fluent-bit.md @@ -9,7 +9,7 @@ Telemetry data processing can be complex, especially at scale. That's why [Fluen The Fluentd and Fluent Bit projects are both: - Licensed under the terms of Apache License v2.0. -- Graduated hosted projects by the [Cloud Native Computing Foundation (CNCF)](https://cncf.io). +- Graduated hosted projects by the [Cloud Native Computing Foundation (CNCF)](https://www.cncf.io/). - Production-grade solutions, with Fluent Bit deployed over 15 billion times globally. - Vendor neutral and community driven. - Widely adopted by the industry, being trusted by major companies like Amazon, Microsoft, Google, and hundreds of others. diff --git a/administration/monitoring.md b/administration/monitoring.md index a2070ffae..f29d77c2f 100644 --- a/administration/monitoring.md +++ b/administration/monitoring.md @@ -469,7 +469,7 @@ When querying the related metrics, the aliases are returned instead of the plugi You can create Grafana dashboards and alerts using Fluent Bit exposed Prometheus style metrics. -The provided [example dashboard](https://github.com/fluent/fluent-bit-docs/blob/master/monitoring/dashboard.json) is heavily inspired by [Banzai Cloud](https://github.com/banzaicloud)'s [logging operator dashboard](https://grafana.com/grafana/dashboards/7752-logging-dashboard) with a few key differences, such as the use of the `instance` label, stacked graphs, and a focus on Fluent Bit metrics. See [this blog post](https://www.robustperception.io/controlling-the-instance-label/) for more information. +The provided [example dashboard](https://github.com/fluent/fluent-bit-docs/blob/master/monitoring/dashboard.json) is heavily inspired by [Banzai Cloud](https://github.com/banzaicloud)'s [logging operator dashboard](https://grafana.com/grafana/dashboards/7752-logging-dashboard/) with a few key differences, such as the use of the `instance` label, stacked graphs, and a focus on Fluent Bit metrics. See [this blog post](https://www.robustperception.io/controlling-the-instance-label/) for more information. ![dashboard](/.gitbook/assets/dashboard.png) diff --git a/installation/downloads.md b/installation/downloads.md index d29b2d4df..c46fab25e 100644 --- a/installation/downloads.md +++ b/installation/downloads.md @@ -50,4 +50,4 @@ Fluent Bit can run on Berkeley Software Distribution (BSD) systems and IBM Z Lin ## Enterprise providers -Fluent Bit packages are also provided by [enterprise providers](https://fluentbit.io/enterprise) for older end-of-life versions, Unix systems, or for additional support and features including aspects (such as CVE backporting). +Fluent Bit packages are also provided by [enterprise providers](https://fluentbit.io/enterprise/) for older end-of-life versions, Unix systems, or for additional support and features including aspects (such as CVE backporting). diff --git a/pipeline/filters/aws-metadata.md b/pipeline/filters/aws-metadata.md index c93e0152f..77af395b4 100644 --- a/pipeline/filters/aws-metadata.md +++ b/pipeline/filters/aws-metadata.md @@ -110,7 +110,7 @@ To achieve this, AWS Filter can be configured with `tags_enabled true` to enable ### Requirements -To use the `tags_enabled true` feature in Fluent Bit, the [instance-metadata-tags](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/modify-instance-metadata-options.html) option must be enabled on the EC2 instance where Fluent Bit is running. Without this option enabled, Fluent Bit won't be able to retrieve the tags associated with the EC2 instance. However, this doesn't mean that Fluent Bit will fail or stop working altogether. Instead, if [instance-metadata-tags](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/modify-instance-metadata-options.html) option isn't enabled, Fluent Bit will continue to operate normally and capture other values, such as the EC2 instance ID or availability zone, based on its configuration. +To use the `tags_enabled true` feature in Fluent Bit, the [`instance-metadata-tags`](https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-metadata-options.html) option must be enabled on the EC2 instance where Fluent Bit is running. Without this option enabled, Fluent Bit won't be able to retrieve the tags associated with the EC2 instance. However, this doesn't mean that Fluent Bit will fail or stop working altogether. Instead, if the `instance-metadata-tags` option isn't enabled, Fluent Bit will continue to operate normally and capture other values, such as the EC2 instance ID or availability zone, based on its configuration. ### Example diff --git a/pipeline/filters/kubernetes.md b/pipeline/filters/kubernetes.md index bf92a47b0..3ed6d2e80 100644 --- a/pipeline/filters/kubernetes.md +++ b/pipeline/filters/kubernetes.md @@ -680,4 +680,4 @@ Learn how to solve them to ensure that the Fluent Bit Kubernetes filter is opera ## Credit -The Kubernetes Filter plugin is fully inspired by the [Fluentd Kubernetes Metadata Filter](https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter) written by [`Jimmi Dyson`](https://github.com/jimmidyson). +The Kubernetes Filter plugin is fully inspired by the [Fluentd Kubernetes Metadata Filter](https://github.com/fluent-plugins-nursery/fluent-plugin-kubernetes_metadata_filter) written by [`Jimmi Dyson`](https://github.com/jimmidyson). diff --git a/pipeline/inputs/cpu-metrics.md b/pipeline/inputs/cpu-metrics.md index 4077e3d5d..765afcb01 100644 --- a/pipeline/inputs/cpu-metrics.md +++ b/pipeline/inputs/cpu-metrics.md @@ -54,7 +54,7 @@ The command returns results similar to the following: ... ``` -As described previously, the CPU input plugin gathers the overall usage every one second and flushed the information to the output on the fifth second. This example uses the `stdout` plugin to demonstrate the output records. In a real use-case you might want to flush this information to some central aggregator such as [Fluentd](https://fluentd.org) or [Elasticsearch](https://elastic.co). +As described previously, the CPU input plugin gathers the overall usage every one second and flushed the information to the output on the fifth second. This example uses the `stdout` plugin to demonstrate the output records. In a real use-case you might want to flush this information to some central aggregator such as [Fluentd](https://www.fluentd.org/) or [Elasticsearch](https://www.elastic.co/). ### Configuration file diff --git a/pipeline/inputs/kafka.md b/pipeline/inputs/kafka.md index 4abe2bc9d..7ce18d91c 100644 --- a/pipeline/inputs/kafka.md +++ b/pipeline/inputs/kafka.md @@ -4,7 +4,7 @@ The _Kafka_ input plugin enables Fluent Bit to consume messages directly from on Starting with version 4.0.4, the Kafka input plugin supports authentication with AWS MSK IAM, enabling integration with Amazon MSK (Managed Streaming for Apache Kafka) clusters that require IAM-based access. -This plugin uses the official [librdkafka C library](https://github.com/edenhill/librdkafka) as a built-in dependency. +This plugin uses the official [librdkafka C library](https://github.com/confluentinc/librdkafka) as a built-in dependency. ## Configuration parameters diff --git a/pipeline/inputs/windows-event-log-winevtlog.md b/pipeline/inputs/windows-event-log-winevtlog.md index 633d11afc..a7f3b0181 100644 --- a/pipeline/inputs/windows-event-log-winevtlog.md +++ b/pipeline/inputs/windows-event-log-winevtlog.md @@ -76,7 +76,7 @@ The default value of `Read_Limit_Per_Cycle` is `512KiB`. #### Query languages for `Event_Query` parameter The `Event_Query` parameter can be used to specify the XML query for filtering Windows EventLog during collection. -The supported query types are [`XPath`](https://developer.mozilla.org/en-US/docs/Web/XPath) and XML Query. +The supported query types are [`XPath`](https://developer.mozilla.org/en-US/docs/Web/XML/XPath) and XML Query. For further details, refer to [Microsoft's documentation](https://learn.microsoft.com/en-us/windows/win32/wes/consuming-events). ### Command line diff --git a/pipeline/outputs/bigquery.md b/pipeline/outputs/bigquery.md index 78b225798..2f2430d6b 100644 --- a/pipeline/outputs/bigquery.md +++ b/pipeline/outputs/bigquery.md @@ -1,26 +1,26 @@ # Google Cloud BigQuery -The _Google Cloud BigQuery_ output plugin is an experimental plugin that lets you stream records into the [Google Cloud BigQuery](https://cloud.google.com/bigquery/) service. +The _Google Cloud BigQuery_ output plugin is an experimental plugin that lets you stream records into the [Google Cloud BigQuery](https://cloud.google.com/bigquery) service. The implementation doesn't support the following, which would be expected in a full production version: -- [Application Default Credentials](https://cloud.google.com/docs/authentication/production). -- [Data deduplication](https://cloud.google.com/bigquery/streaming-data-into-bigquery) using `insertId`. -- [Template tables](https://cloud.google.com/bigquery/streaming-data-into-bigquery) using `templateSuffix`. +- [Application Default Credentials](https://docs.cloud.google.com/docs/authentication). +- [Data deduplication](https://docs.cloud.google.com/bigquery/docs/streaming-data-into-bigquery) using `insertId`. +- [Template tables](https://docs.cloud.google.com/bigquery/docs/streaming-data-into-bigquery) using `templateSuffix`. ## Google Cloud configuration Fluent Bit streams data into an existing BigQuery table using a service account that you specify. Before using the BigQuery output plugin, you must: -1. To stream data into BigQuery, you must create a [Google Cloud service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts) for Fluent Bit. +1. To stream data into BigQuery, you must create a [Google Cloud service account](https://docs.cloud.google.com/iam/docs/service-accounts-create) for Fluent Bit. 1. Create a BigQuery dataset. - Fluent Bit doesn't create datasets for your data, so you must [create the dataset](https://cloud.google.com/bigquery/docs/datasets) ahead of time. You must also grant the service account `WRITER` permission on the dataset. + Fluent Bit doesn't create datasets for your data, so you must [create the dataset](https://docs.cloud.google.com/bigquery/docs/datasets) ahead of time. You must also grant the service account `WRITER` permission on the dataset. Within the dataset you must create a table for the data to reside in. Use the following instructions for creating your table. Pay close attention to the schema, as it must match the schema of your output JSON. Unfortunately, because BigQuery doesn't allow dots in field names, you must use a filter to change the fields for many of the standard inputs (for example, `mem` or `cpu`). -1. [Create a BigQuery table](https://cloud.google.com/bigquery/docs/tables). -1. Fluent Bit BigQuery output plugin uses a JSON credentials file for authentication credentials. [Authorize the service account](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) to write to the table. +1. [Create a BigQuery table](https://docs.cloud.google.com/bigquery/docs/tables). +1. Fluent Bit BigQuery output plugin uses a JSON credentials file for authentication credentials. [Authorize the service account](https://docs.cloud.google.com/iam/docs/keys-create-delete) to write to the table. 1. Provide the service account credentials to Fluent Bit. - With [workload identity federation](https://cloud.google.com/iam/docs/workload-identity-federation), you can grant on-premises or multi-cloud workloads access to Google Cloud resources, without using a service account key. It can be used as a more secure alternative to service account credentials. Google Cloud's workload identity federation supports several identity providers (see documentation) but Fluent Bit BigQuery plugin currently supports Amazon Web Services (AWS) only. + With [workload identity federation](https://docs.cloud.google.com/iam/docs/workload-identity-federation), you can grant on-premises or multi-cloud workloads access to Google Cloud resources, without using a service account key. It can be used as a more secure alternative to service account credentials. Google Cloud's workload identity federation supports several identity providers (see documentation) but Fluent Bit BigQuery plugin currently supports Amazon Web Services (AWS) only. You must configure workload identity federation in GCP before using it with Fluent Bit. @@ -45,7 +45,7 @@ Fluent Bit streams data into an existing BigQuery table using a service account | `google_service_account` | The email address of the Google service account to impersonate. The workload identity provider must have permissions to impersonate this service account, and the service account must have permissions to access Google BigQuery resources ( `write` access to tables) | _none_ | | `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | -See Google's [official documentation](https://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll) for further details. +See Google's [official documentation](https://docs.cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll) for further details. ## Configuration file diff --git a/pipeline/outputs/chronicle.md b/pipeline/outputs/chronicle.md index 8f5bf4104..872668e1d 100644 --- a/pipeline/outputs/chronicle.md +++ b/pipeline/outputs/chronicle.md @@ -1,6 +1,6 @@ # Google Chronicle -The _Google Chronicle_ output plugin lets you ingest security logs into the [Google Chronicle](https://chronicle.security/) service. This connector is designed to send unstructured security logs. +The _Google Chronicle_ output plugin lets you ingest security logs into the [Google Chronicle](https://cloud.google.com/security/products/security-operations) service. This connector is designed to send unstructured security logs. ## Google Cloud configuration @@ -8,7 +8,7 @@ Fluent Bit streams data into an existing Google Chronicle tenant using a service 1. Create a service account. - To stream security logs into Google Chronicle, create a [Google Cloud service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts) for Fluent Bit: + To stream security logs into Google Chronicle, create a [Google Cloud service account](https://docs.cloud.google.com/iam/docs/service-accounts-create) for Fluent Bit: 1. Create a tenant of Google Chronicle. @@ -16,7 +16,7 @@ Fluent Bit streams data into an existing Google Chronicle tenant using a service 1. Retrieve service account credentials. - The Fluent Bit Chronicle output plugin uses a JSON credentials file for authentication credentials. Download the credentials file by following the instructions for [Creating and Managing Service Account Keys](https://cloud.google.com/iam/docs/creating-managing-service-account-keys). + The Fluent Bit Chronicle output plugin uses a JSON credentials file for authentication credentials. Download the credentials file by following the instructions for [Creating and Managing Service Account Keys](https://docs.cloud.google.com/iam/docs/keys-create-delete). ## Configurations parameters @@ -27,7 +27,7 @@ Fluent Bit streams data into an existing Google Chronicle tenant using a service | `service_account_secret` | Private key content associated with the service account. Only available if no credentials file has been provided. | Value of environment variable `$SERVICE_ACCOUNT_SECRET` | | `project_id` | The project id containing the tenant of Google Chronicle to stream into. | The value of the `project_id` in the credentials file | | `customer_id` | The customer id to identify the tenant of Google Chronicle to stream into. The value of the `customer_id` should be specified in the configuration file. | _none_ | -| `log_type` | The log type to parse logs as. Google Chronicle supports parsing for [specific log types only](https://cloud.google.com/chronicle/docs/ingestion/parser-list/supported-default-parsers). | _none_ | +| `log_type` | The log type to parse logs as. Google Chronicle supports parsing for [specific log types only](https://docs.cloud.google.com/chronicle/docs/ingestion/parser-list/supported-default-parsers). | _none_ | | `region` | The GCP region in which to store security logs. Supported regions: `US`, `EU`, `UK`, `ASIA`. Blank is handled as `US`. | _none_ | | `log_key` | By default, the whole log record will be sent to Google Chronicle. If you specify a key name with this option, then only the value of that key will be sent to Google Chronicle. | _none_ | | `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | diff --git a/pipeline/outputs/cloudwatch.md b/pipeline/outputs/cloudwatch.md index a262d227b..0539302b1 100644 --- a/pipeline/outputs/cloudwatch.md +++ b/pipeline/outputs/cloudwatch.md @@ -407,7 +407,7 @@ If you see errors for image pull limits, try log into public ECR with your AWS c aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws ``` -You can check the [Amazon ECR Public official doc](https://docs.aws.amazon.com/AmazonECR/latest/public/get-set-up-for-amazon-ecr.html) for more details +You can check the [Amazon ECR Public official doc](https://docs.aws.amazon.com/AmazonECR/latest/public/what-is-ecr.html) for more details #### Docker Hub diff --git a/pipeline/outputs/datadog.md b/pipeline/outputs/datadog.md index 09170ab23..8e90d6ab8 100644 --- a/pipeline/outputs/datadog.md +++ b/pipeline/outputs/datadog.md @@ -4,9 +4,9 @@ description: Send logs to Datadog # Datadog -The _Datadog_ output plugin lets you ingest your logs into [Datadog](https://app.datadoghq.com/signup). +The _Datadog_ output plugin lets you ingest your logs into [Datadog](https://www.datadoghq.com/). -Before you begin, you need a [Datadog account](https://app.datadoghq.com/signup), a [Datadog API key](https://docs.datadoghq.com/account_management/api-app-keys/), and you need to [activate Datadog Logs Management](https://app.datadoghq.com/logs/activation). +Before you begin, you need a [Datadog account](https://app.datadoghq.com/signup), a [Datadog API key](https://docs.datadoghq.com/account_management/api-app-keys/), and you need to [activate Datadog Logs Management](https://app.datadoghq.com/account/login?next=%2Flogs%2Factivation). ## Configuration parameters @@ -16,7 +16,7 @@ This plugin uses the following configuration parameters: | --- | ----------- | ------- | | `Host` | The Datadog server where you are sending your logs. | `http-intake.logs.datadoghq.com` | | `TLS` | End-to-end security communications security protocol. Datadog recommends setting this to `on`. | `off` | -| `compress` | Optional. Compresses the payload in GZIP format. Datadog supports and recommends setting this to `gzip`. | _none_ | +| `compress` | Optional. Compresses the payload in GZIP format. Datadog supports and recommends setting this to `gzip`. | _none_ | | `apikey` | Your [Datadog API key](https://app.datadoghq.com/account/settings#api). | _none_ | | `Proxy` | Optional. Specifies an HTTP proxy. The expected format of this value is `http://host:port`. HTTPS isn't supported. | _none_ | | `provider` | To activate remapping, specify the configuration flag provider with the value `ecs`. | _none_ | @@ -25,7 +25,7 @@ This plugin uses the following configuration parameters: | `tag_key` | The key name of tag. If `include_tag_key` is `false`, this property is ignored. | `tagkey` | | `dd_service` | Recommended. The human readable name for your service generating the logs. For example, the name of your application or database. If not set, Datadog looks for the service using [service remapper](https://docs.datadoghq.com/logs/log_configuration/pipelines/?tab=service#service-attribute). | _none_ | | `dd_source` | Recommended. A human-readable name for the underlying technology of your service like `postgres` or `nginx`. If unset, Datadog looks for the source in the [`ddsource` attribute](https://docs.datadoghq.com/logs/log_configuration/pipelines/?tab=source#source-attribute). | _none_ | -| `dd_tags` | Optional. The [tags](https://docs.datadoghq.com/tagging/) you want to assign to your logs in Datadog. If unset, Datadog will look for the tags in the [`ddtags` attribute](https://docs.datadoghq.com/api/latest/logs/#send-logs). | _none_ | +| `dd_tags` | Optional. The [tags](https://docs.datadoghq.com/getting_started/tagging/) you want to assign to your logs in Datadog. If unset, Datadog will look for the tags in the [`ddtags` attribute](https://docs.datadoghq.com/api/latest/logs/#send-logs). | _none_ | | `dd_message_key` | By default, the plugin searches for the key `log` and remaps the value to the key `message`. If the property is set, the plugin will search the property name key. | _none_ | | `dd_hostname` | The host the emitted logs should be associated with. If unset, Datadog expects the host to be set with `host`, `hostname`, or `syslog.hostname` attributes. See [Datadog Logs preprocessor documentation](https://docs.datadoghq.com/logs/log_configuration/pipelines/?tab=host#preprocessing) for recognized attributes. | _none_ | | workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | diff --git a/pipeline/outputs/dynatrace.md b/pipeline/outputs/dynatrace.md index 3912b05cb..dd9bcb785 100644 --- a/pipeline/outputs/dynatrace.md +++ b/pipeline/outputs/dynatrace.md @@ -4,7 +4,7 @@ description: Send logs to Dynatrace # Dynatrace -Stream logs to [Dynatrace](https://www.dynatrace.com) by utilizing the `http` plugin to send data to the [Dynatrace generic log ingest API](https://docs.dynatrace.com/docs/shortlink/lma-generic-log-ingestion). +Stream logs to [Dynatrace](https://www.dynatrace.com) by utilizing the `http` plugin to send data to the [Dynatrace generic log ingest API](https://docs.dynatrace.com/docs/analyze-explore-automate/logs/lma-log-ingestion/lma-log-ingestion-via-api). ## Configuration parameters @@ -26,7 +26,7 @@ Stream logs to [Dynatrace](https://www.dynatrace.com) by utilizing the `http` pl To get started with sending logs to Dynatrace: -1. Get a [Dynatrace API](https://docs.dynatrace.com/docs/shortlink/api-authentication) token with the `logs.ingest` (Ingest Logs) scope. +1. Get a [Dynatrace API](https://docs.dynatrace.com/docs/dynatrace-api/basics/dynatrace-api-authentication) token with the `logs.ingest` (Ingest Logs) scope. 1. Determine your Dynatrace [environment ID](https://docs.dynatrace.com/docs/shortlink/monitoring-environment#environment-id). 1. In your main Fluent Bit configuration file, append the following `Output` section: @@ -83,7 +83,6 @@ To get started with sending logs to Dynatrace: - [Dynatrace Fluent Bit documentation](https://docs.dynatrace.com/docs/shortlink/lma-stream-logs-with-fluent-bit) - [Fluent Bit integration in Dynatrace Hub](https://www.dynatrace.com/hub/detail/fluent-bit/?filter=log-management-and-analytics) - [Video: Stream a Log File to Dynatrace using Fluent Bit](https://www.youtube.com/watch?v=JJJNxhtJ6R0) -- [Blog: Easily stream logs from Fluent Bit to - Dynatrace](https://www.dynatrace.com/news/blog/easily-stream-logs-with-fluent-bit-to-dynatrace/) +- [Blog: Easily stream logs from Fluent Bit to Dynatrace](https://docs.dynatrace.com/docs/analyze-explore-automate/logs/lma-log-ingestion/lma-stream-logs-with-fluent-bit) diff --git a/pipeline/outputs/elasticsearch.md b/pipeline/outputs/elasticsearch.md index 0c21f31f1..3e9a98a0a 100644 --- a/pipeline/outputs/elasticsearch.md +++ b/pipeline/outputs/elasticsearch.md @@ -49,7 +49,7 @@ This plugin has the following configuration parameters: | `Trace_Output` | Print all ElasticSearch API request payloads to `stdout` for diagnostics. | `Off` | | `Trace_Error` | If ElasticSearch returns an error, print the ElasticSearch API request and response for diagnostics. | `Off` | | `Current_Time_Index` | Use current time for index generation instead of message record. | `Off` | -| `Suppress_Type_Name` | When enabled, mapping types is removed and `Type` option is ignored. Elasticsearch 8.0.0 or higher [no longer supports mapping types](https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html), and is set to `On`. | `Off` | +| `Suppress_Type_Name` | When enabled, mapping types is removed and `Type` option is ignored. Elasticsearch 8.0.0 or higher [no longer supports mapping types](https://www.elastic.co/docs/manage-data/data-store/mapping/removal-of-mapping-types), and is set to `On`. | `Off` | | `Workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `2` | If you have used a common relational database, the parameters `index` and `type` can be compared to the `database` and `table` concepts. @@ -171,7 +171,7 @@ Connect to Amazon OpenSearch or Elastic Cloud with the ElasticSearch plugin. The Amazon OpenSearch Service adds an extra security layer where HTTP requests must be signed with AWS Sigv4. Fluent Bit v1.5 introduced full support for Amazon OpenSearch Service with IAM Authentication. -See [details](https://github.com/fluent/fluent-bit-docs/tree/43c4fe134611da471e706b0edb2f9acd7cdfdbc3/administration/aws-credentials.md) on how AWS credentials are fetched. +See [details](../../administration/aws-credentials.md) on how AWS credentials are fetched. Example configuration: @@ -216,7 +216,8 @@ Be aware that the `Port` is set to `443`, `tls` is enabled, and `AWS_Region` is ### Use Fluent Bit with Elastic Cloud -Fluent Bit supports connecting to [Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-getting-started.html) by providing the `cloud_id` and the `cloud_auth` settings. `cloud_auth` uses the `elastic` user and password provided when the cluster was created. For details refer to the [Cloud ID usage page](https://www.elastic.co/guide/en/cloud/current/ec-cloud-id.html). +Fluent Bit supports connecting to [Elastic Cloud](https://www.elastic.co/docs/deploy-manage/deploy/elastic-cloud/cloud-hosted) by providing the `cloud_id` and the `cloud_auth` settings. `cloud_auth` uses the `elastic` user and password provided when the cluster was created. For details refer to the [Cloud ID usage page](https://www.elastic.co/docs/deploy-manage/deploy/elastic-cloud/find-cloud-id). + Example configuration: {% tabs %} diff --git a/pipeline/outputs/firehose.md b/pipeline/outputs/firehose.md index 0f861817f..bae02bd4d 100644 --- a/pipeline/outputs/firehose.md +++ b/pipeline/outputs/firehose.md @@ -4,7 +4,7 @@ description: Send logs to Amazon Kinesis Firehose # Amazon Kinesis Data Firehose -The _Amazon Kinesis Data Firehose_ output plugin lets you ingest your records into the [Firehose](https://aws.amazon.com/kinesis/data-firehose/) service. +The _Amazon Kinesis Data Firehose_ output plugin lets you ingest your records into the [Firehose](https://aws.amazon.com/firehose/) service. This is the documentation for the core Fluent Bit Firehose plugin written in C. It can replace the [aws/amazon-kinesis-firehose-for-fluent-bit](https://github.com/aws/amazon-kinesis-firehose-for-fluent-bit) Golang Fluent Bit plugin. The Golang plugin was named `firehose`. This new Firehose plugin is called `kinesis_firehose` to prevent conflicts/confusion. @@ -162,7 +162,7 @@ If you see errors for image pull limits, try log into public ECR with your AWS c aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws ``` -You can check the [Amazon ECR Public official doc](https://docs.aws.amazon.com/AmazonECR/latest/public/get-set-up-for-amazon-ecr.html) for more details. +You can check the [Amazon ECR Public official doc](https://docs.aws.amazon.com/AmazonECR/latest/public/) for more details. #### Docker Hub diff --git a/pipeline/outputs/gelf.md b/pipeline/outputs/gelf.md index 7bef1bb1c..8da3b91a3 100644 --- a/pipeline/outputs/gelf.md +++ b/pipeline/outputs/gelf.md @@ -1,6 +1,6 @@ # Graylog Extended Log Format (GELF) -The _[Graylog](https://www.graylog.org) Extended Log Format (GELF)_ output plugin lets you send logs in GELF format directly to a Graylog input using TLS, TCP, or UDP protocols. +The _[Graylog](https://graylog.org) Extended Log Format (GELF)_ output plugin lets you send logs in GELF format directly to a Graylog input using TLS, TCP, or UDP protocols. The following instructions assume that you have a fully operational Graylog server running in your environment. diff --git a/pipeline/outputs/influxdb.md b/pipeline/outputs/influxdb.md index aeb560780..a1bc2c55b 100644 --- a/pipeline/outputs/influxdb.md +++ b/pipeline/outputs/influxdb.md @@ -1,6 +1,6 @@ # InfluxDB -The _InfluxDB_ output plugin lets you flush your records into a [InfluxDB](https://www.influxdata.com/time-series-platform/influxdb/) time series database. The following instructions assume that you have an operational InfluxDB service running in your system. +The _InfluxDB_ output plugin lets you flush your records into a [InfluxDB](https://www.influxdata.com/products/influxdb-overview) time series database. The following instructions assume that you have an operational InfluxDB service running in your system. ## Configuration parameters diff --git a/pipeline/outputs/kafka.md b/pipeline/outputs/kafka.md index 6ef0b56fe..4ad785e18 100644 --- a/pipeline/outputs/kafka.md +++ b/pipeline/outputs/kafka.md @@ -1,6 +1,6 @@ # Kafka Producer -The _Kafka Producer_ output plugin lets you ingest your records into an [Apache Kafka](https://kafka.apache.org/) service. This plugin uses the official [librdkafka C library](https://github.com/edenhill/librdkafka). +The _Kafka Producer_ output plugin lets you ingest your records into an [Apache Kafka](https://kafka.apache.org/) service. This plugin uses the official [librdkafka C library](https://github.com/confluentinc/librdkafka). In Fluent Bit 4.0.4 and later, the Kafka input plugin supports authentication with AWS MSK IAM, enabling integration with Amazon MSK (Managed Streaming for Apache Kafka) clusters that require IAM-based access. @@ -20,7 +20,7 @@ This plugin supports the following parameters: | `topic_key` | If multiple `topics` exist, the value of `Topic_Key` in the record will indicate the topic to use. For example, if `Topic_Key` is `router` and the record is `{"key1": 123, "router": "route_2"}`, Fluent Bit will use `topic _route_2_`. If the value of `Topic_Key` isn't present in `topics`, then the first topic in the `topics` list will indicate the topic to be used. | _none_ | | `dynamic_topic` | Adds unknown topics (found in `Topic_Key`) to `topics`. In `topics`, only a default topic needs to be configured. | `Off` | | `queue_full_retries` | Fluent Bit queues data into `rdkafka` library. If the underlying library can't flush the records the queue might fill up, blocking new addition of records. `queue_full_retries` sets the number of local retries to enqueue the data. The interval between retries is 1 second. Setting the `queue_full_retries` value to `0` sets an unlimited number of retries. | `10` | -| `rdkafka.{property}` | `{property}` can be any [librdkafka properties](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) | | +| `rdkafka.{property}` | `{property}` can be any [librdkafka properties](https://github.com/confluentinc/librdkafka/blob/master/CONFIGURATION.md) | _none_ | | `raw_log_key` | When using the raw format and set, the value of `raw_log_key` in the record will be send to Kafka as the payload. | _none_ | | `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | diff --git a/pipeline/outputs/loki.md b/pipeline/outputs/loki.md index 0343dcbd9..c51843793 100644 --- a/pipeline/outputs/loki.md +++ b/pipeline/outputs/loki.md @@ -4,7 +4,7 @@ The Fluent Bit _Loki_ built-in output plugin lets you send your log or events to a Loki service. It supports data enrichment with Kubernetes labels, custom label keys, and Tenant ID, along with other information. -There is a separate Golang output plugin provided by [Grafana](https://grafana.com/docs/loki/latest/clients/fluentbit/) with different configuration options. +There is a separate Golang output plugin provided by [Grafana](https://grafana.com/docs/loki/latest/send-data/fluentbit/) with different configuration options. ## Configuration parameters diff --git a/pipeline/outputs/observe.md b/pipeline/outputs/observe.md index b162add17..96f3c385a 100644 --- a/pipeline/outputs/observe.md +++ b/pipeline/outputs/observe.md @@ -1,6 +1,6 @@ # Observe -Use the [HTTP output plugin](./http.md) to flush your records [into Observe](https://docs.observeinc.com/en/latest/content/data-ingestion/forwarders/fluentbit.html). It issues a POST request with the data records in [MessagePack](https://msgpack.org) (or JSON) format. +Use the [HTTP output plugin](./http.md) to flush your records [into Observe](https://docs.observeinc.com/docs/fluent-bit). It issues a POST request with the data records in [MessagePack](https://msgpack.org) (or JSON) format. ## Configuration parameters @@ -8,7 +8,7 @@ The following HTTP configuration parameters are relevant to Observe: | Key | Description | Default | | --- | ----------- | ------- | -| `host` | IP address or hostname of the Observe data collection endpoint. Replace `$(OBSERVE_CUSTOMER)` with your [Customer ID](https://docs.observeinc.com/en/latest/content/hints/CustomerId.html). | `OBSERVE_CUSTOMER.collect.observeinc.com` | +| `host` | IP address or hostname of the Observe data collection endpoint. Replace `$(OBSERVE_CUSTOMER)` with your [Customer ID](https://docs.observeinc.com/docs/where-do-i-find-my-customer-id). | `OBSERVE_CUSTOMER.collect.observeinc.com` | | `port` | TCP port to use when sending data to Observe. | `443` | | `tls` | Specifies whether to use TLS. | `on` | | `uri` | Specifies the HTTP URI for Observe. | `/v1/http/fluentbit` | diff --git a/pipeline/outputs/openobserve.md b/pipeline/outputs/openobserve.md index 83a85aa5d..e50c3a105 100644 --- a/pipeline/outputs/openobserve.md +++ b/pipeline/outputs/openobserve.md @@ -7,9 +7,9 @@ description: Send logs to OpenObserve using Fluent Bit Use the OpenObserve output plugin to ingest logs into [OpenObserve](https://openobserve.ai/). -Before you begin, you need the following fields, which can be found under **Ingestion** in OpenObserve Cloud. Alternatively, you can achieve this with various installation types as mentioned in the [OpenObserve documentation](https://openobserve.ai/docs/quickstart/): +Before you begin, you need the following fields, which can be found under **Ingestion** in OpenObserve Cloud. Alternatively, you can achieve this with various installation types as mentioned in the [OpenObserve documentation](https://openobserve.ai/docs/getting-started/): -- [OpenObserve account](https://cloud.openobserve.ai/) +- [OpenObserve account](https://cloud.openobserve.ai/web/) - `HTTP_User` - `HTTP_Passwd` diff --git a/pipeline/outputs/parseable.md b/pipeline/outputs/parseable.md index 7fe636af0..ea35a04ae 100644 --- a/pipeline/outputs/parseable.md +++ b/pipeline/outputs/parseable.md @@ -153,4 +153,4 @@ Parseable supports HTTP basic authentication. Configure the `http_user` and `htt ## References - [Parseable documentation](https://www.parseable.com/docs) -- [Parseable Fluent-bit integration](https://www.parseable.com/docs/datasource/log-agents/fluent-bit) +- [Parseable Fluent-bit integration](https://www.parseable.com/docs/ingest-data/logging-agents/fluent-bit) diff --git a/pipeline/outputs/prometheus-remote-write.md b/pipeline/outputs/prometheus-remote-write.md index 8994eb418..370e3fd25 100644 --- a/pipeline/outputs/prometheus-remote-write.md +++ b/pipeline/outputs/prometheus-remote-write.md @@ -13,7 +13,7 @@ The Prometheus exporter works only with metric plugins, such as Node Exporter Me This plugin supports the following parameters: | Key | Description | Default | -|-----|-------------|---------| +| --- | ----------- | ------- | | `host` | IP address or hostname of the target HTTP server. | `127.0.0.1` | | `http_user` | Basic Auth username. | _none_ | | `http_passwd` | Basic Auth Password. Requires `HTTP_user` to be set. | _none_ | @@ -159,7 +159,7 @@ pipeline: ### Logz.io infrastructure monitoring -With Logz.io [hosted Prometheus](https://logz.io/solutions/infrastructure-monitoring/) you will need to make use of the header option and add the Authorization Bearer with the proper key. The host and port can also differ within your specific hosted instance. +With Logz.io [hosted Prometheus](https://logz.io/platform/infrastructure-monitoring/) you will need to make use of the header option and add the Authorization Bearer with the proper key. The host and port can also differ within your specific hosted instance. {% tabs %} {% tab title="fluent-bit.yaml" %} @@ -236,7 +236,7 @@ pipeline: ### Levitate -With [Levitate](https://last9.io/levitate-tsdb), you must use the Levitate cluster-specific write URL and specify the HTTP username and password for the token created for your Levitate cluster. +With [Levitate](https://last9.io/), you must use the Levitate cluster-specific write URL and specify the HTTP username and password for the token created for your Levitate cluster. {% tabs %} {% tab title="fluent-bit.yaml" %} diff --git a/pipeline/outputs/s3.md b/pipeline/outputs/s3.md index e325660a2..cba4f1e8f 100644 --- a/pipeline/outputs/s3.md +++ b/pipeline/outputs/s3.md @@ -8,13 +8,13 @@ description: Send logs, data, and metrics to Amazon S3 The _Amazon S3_ output plugin lets you ingest records into the [S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) cloud object store. -The plugin can upload data to S3 using the [multipart upload API](https://docs.aws.amazon.com/AmazonS3/latest/dev/uploadobjusingmpu.html) or [`PutObject`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html). Multipart is the default and is recommended. Fluent Bit will stream data in a series of _parts_. This limits the amount of data buffered on disk at any point in time. By default, every time 5 MiB of data have been received, a new part will be uploaded. The plugin can create files up to gigabytes in size from many small chunks or parts using the multipart API. All aspects of the upload process are configurable. +The plugin can upload data to S3 using the [multipart upload API](https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html) or [`PutObject`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html). Multipart is the default and is recommended. Fluent Bit will stream data in a series of _parts_. This limits the amount of data buffered on disk at any point in time. By default, every time 5 MiB of data have been received, a new part will be uploaded. The plugin can create files up to gigabytes in size from many small chunks or parts using the multipart API. All aspects of the upload process are configurable. The plugin lets you specify a maximum file size, and a timeout for uploads. A file will be created in S3 when the maximum size or the timeout is reached, whichever comes first. Records are stored in files in S3 as newline delimited JSON. -See [AWS Credentials](https://github.com/fluent/fluent-bit-docs/tree/43c4fe134611da471e706b0edb2f9acd7cdfdbc3/administration/aws-credentials.md) +See [AWS Credentials](../../administration/aws-credentials.md) for details about fetching AWS credentials. {% hint style="warning" %} @@ -32,80 +32,80 @@ The [Prometheus success/retry/error metrics values](../../administration/monitor ## Configuration parameters | Key | Description | Default | -|-----|-------------|----------| -| `alias` | Sets an alias, use for multiple instances of the same output plugin. | _none_ | -| `authorization_endpoint_bearer_token` | Authorization endpoint bearer token. | _none_ | -| `authorization_endpoint_password` | Authorization endpoint basic authentication password.| _none_ | -| `authorization_endpoint_url` | Authorization endpoint URL. | _none_ | -| `authorization_endpoint_username` | Authorization endpoint basic authentication username.| _none_ | -| `auto_retry_requests` | Immediately retry failed requests to AWS services once. This option doesn't affect the normal Fluent Bit retry mechanism with backoff. Instead, it enables an immediate retry with no delay for networking errors, which can help improve throughput during transient network issues. | `true` | -| `blob_database_file` | Absolute path to a database file to be used to store blob files contexts. | _none_ | -| `bucket` | S3 Bucket name | _none_ | -| `canned_acl` | [Predefined Canned ACL policy](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) for S3 objects. | _none_ | -| `compression` | Compression/format for S3 objects. Supported: `gzip` (always available) and `parquet` (requires Arrow build). For `gzip`, the `Content-Encoding` header is set to `gzip`. `parquet` is available **only when Fluent Bit is built with `-DFLB_ARROW=On`** and Arrow GLib/Parquet GLib are installed. Parquet is typically used with `use_put_object On`. | _none_ | -| `content_type` | A standard MIME type for the S3 object, set as the Content-Type HTTP header. | _none_ | -| `endpoint` | Custom endpoint for the S3 API. Endpoints can contain scheme and port. | _none_ | -| `external_id` | Specify an external ID for the STS API. Can be used with the `role_arn` parameter if your role requires an external ID. | _none_ | -| `file_delivery_attempt_limit` | File delivery attempt limit. | `1` | -| `host` | IP address or hostname of the target HTTP server. | `127.0.0.1` | -| `json_date_format` | Specify the format of the date. Accepted values: `double`, `epoch`, `epoch_ms`, `iso8601` (2018-05-30T09:39:52.000681Z), `_java_sql_timestamp_` (2018-05-30 09:39:52.000681). | _none_ | -| `json_date_key` | Specify the name of the date key in the output record. To disable the time key, set the value to `false`. | `date` | -| `log_level` | Specifies the log level for output plugin. If not set here, plugin uses global log level in `service` section. | `info` | -| `log_supress_interval` | Suppresses log messages from output plugin that appear similar within a specified time interval. `0` no suppression. | `0` | -| `match` | Set a tag pattern to match records that output should process. Exact matches or wildcards (for example `*`). | _none_ | -| `match_regex` | Set a regular expression to match tags for output routing. This allows more flexible matching compared to wildcards. | _none_ | -| `net.connect_timeout` | Set maximum time allowed to establish a connection, this time includes the TLS handshake. | `10s` | -| `net.connect_timeout_log_error` | On connection timeout, specify if it should log an error. When disabled, the timeout is logged as a debug message. | `true` | -| `net.keepalive_max_recycle` | Set maximum number of times a keepalive connection can be used before it retries. | `2000` | -| `net.dns.mode` | Select the primary DNS connection type (TCP or UDP). | _none_ | -| `net.dns.prefer_ipv4` | Select the primary DNS resolver type (LEGACY or ASYNC). | _none_ | -| `net.dns.prefer_ipv6` | Prioritize IPv6 DNS results when trying to establish a connection. | _none_ | -| `net.io_timeout` | Set maximum time a connection can stay idle while assigned. | `0s` | -| `net.max_worker_connections` | Set the maximum number of active TCP connections that can be used per worker thread. | `0` | -| `net.proxy_env_ignore` | Ignore the environment variables `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` when set. | `false` | -| `net.source_address` | Specify network address to bind for data traffic. | _none_ | -| `net.tcp_keepalive` | Enable or disable Keepalive support. | `off` | -| `net.tcp_keepalive_time` | Interval between the last data packet sent and the first TCP keepalive probe. | `-1` | -| `net.tcp_keepalive_interval` | Interval between TCP keepalive probes when no response is received on a `keepidle` probe. | `-1` | -| `net.tcp_keepalive_probes` | Number of unacknowledged probes to consider a connection dead. | `-1` | -| `log_key` | By default, the whole log record will be sent to S3. When specifying a key name with this option, only the value of that key sends to S3. For example, when using Docker you can specify `log_key log` and only the log message sends to S3. | _none_ | -| `part_delivery_attempt_limit` | File part delivery attempt limit. | `1` | -| `part_size` | Size of each part when uploading blob files. | `25M` | -| `port` | TCP port of the target HTTP server. | `80` | -| `preserve_data_ordering` | When an upload request fails, the last received chunk might swap with a later chunk, resulting in data shuffling. This feature prevents shuffling by using a queue logic for uploads. | `true` | -| `profile` | Option to specify an AWS Profile for credentials. | _none_ | -| `region` | The AWS region of your S3 bucket. | `us-east-1` | -| `retry_limit` | Set retry limit for output plugin when delivery fails. Integer, `no_limits`, `false`, or `off` to disable, or `no_retries` to disable retries entirely. | `1` | -| `role_arn` | ARN of an IAM role to assume (for example, for cross account access.) | _none_ | -| `s3_key_format` | Format string for keys in S3. This option supports a UUID, strftime time formatters, a syntax for selecting parts of the Fluent log tag using a syntax inspired by the `rewrite_tag` filter. Add `$UUID` in the format string to insert a random string. Add `$INDEX` in the format string to insert an integer that increments each upload. The `$INDEX` value saves in the `store_dir`. Add `$TAG` in the format string to insert the full log tag. Add `$TAG[0]` to insert the first part of the tag in the S3 key. The tag is split into parts using the characters specified with the `s3_key_format_tag_delimiters` option. Add the extension directly after the last piece of the format string to insert a key suffix. To specify a key suffix in `use_put_object` mode, you must specify `$UUID`. See [S3 Key Format](#s3-key-format-and-tag-delimiters). Time in `s3_key` is the timestamp of the first record in the S3 file. | `/fluent-bit-logs/$TAG/%Y/%m/%d/%H/%M/%S` | -| `s3_key_format_tag_delimiters` | A series of characters which will be used to split the tag into `parts` for use with the s3_key_format option. | `.` | -| `send_content_md5` | Send the Content-MD5 header with `PutObject` and UploadPart requests, as is required when Object Lock is enabled. | `false` | -| `static_file_path` | Disables behavior where UUID string appends to the end of the S3 key name when `$UUID` isn't provided in `s3_key_format`. `$UUID`, time formatters, `$TAG`, and other dynamic key formatters all work as expected while this feature is set to true. | `false` | -| `store_dir` | Directory to locally buffer data before sending. Plugin uses the S3 Multipart upload API to send data in chunks of 5 MB at a time. | `/tmp/fluent-bit/s3` | -| `store_dir_limit_size` | S3 plugin has its own buffering system with files in the `store_dir`. Use the `store_dir_limit_size` to limit the amount of data S3 buffers in the `store_dir` to limit disk usage. If the limit is reached, data will be discarded. Default is 0 which means unlimited. | `0` | -| `storage_class` | Specify the [storage class](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html#AmazonS3-PutObject-request-header-StorageClass) for S3 objects. If this option isn't specified, objects store with the default `STANDARD` storage class. | _none_ | -| `sts_endpoint` | Custom endpoint for the STS API. | _none_ | -| `tls` | Enable or disable TLS/SSL support. | `off` | -| `tls.ca_file` | Absolute path to CA certificate file. | _none_ | -| `tls.ca_path` | Absolute path to scan for certificate files. | _none_ | -| `tls.crt_file` | Absolute path to Certificate file. | _none_ | -| `tls.ciphers` | Specify TLS ciphers up to TLSv1.2. | _none_ | -| `tls.debug` | Set TLS debug level. Accepts `0` (No debug), `1`(Error), `2` (State change), `3` (Informational) and `4` (Verbose). | `1` | -| `tls.key_file` | Absolute path to private Key file. | _none_ | -| `tls.key_passwd` | Optional password for tls.key_file file.| _none_ | -| `tls.max_version` | Specify the maximum version of TLS. | _none_ | -| `tls.min_version` | Specify the minimum version of TLS. | _none_ | -| `tls.verify_hostname` | Enable or disable to verify hostname. | `off` | -| `tls.vhost` | Hostname to be used for TLS SNI extension. | _none_ | -| `tls.verify` | Force certificate validation. | `on` | -| `tls.windows.certstore_name` | Sets the `certstore` name on an output (Windows). | _none_ | -| `tls.windows.use_enterprise_store` | Sets whether using enterprise `certstore` or not on an output (Windows). | _none_ | -| `total_file_size` | Specify file size in S3. Minimum size is `1M`. With `use_put_object On` the maximum size is `1G`. With multipart uploads, the maximum size is `50G`.| `100000000` | -| `upload_chunk_size` | The size of each part for multipart uploads. Default: 5M, Max: 50M, Min: 5M. | `5242880` | -| `upload_part_freshness_timeout` | Maximum lifespan of an uncommitted file part. | `6D` | -| `upload_parts_timeout` | Timeout to upload parts of a blob file. | `10m` | -| `upload_timeout` | When this amount of time elapses, Fluent Bit uploads and creates a new file in S3. Set to `60m` to upload a new file every hour. | `10m` | -| `use_put_object` | Use the S3 `PutObject` API instead of the multipart upload API. When enabled, the key extension is only available when `$UUID` is specified in `s3_key_format`. If `$UUID` isn't included, a random string appends format string and the key extension can't be customized. | `false` | +| --- | ----------- | ------- | +| `alias` | Sets an alias, use for multiple instances of the same output plugin. | _none_ | +| `authorization_endpoint_bearer_token` | Authorization endpoint bearer token. | _none_ | +| `authorization_endpoint_password` | Authorization endpoint basic authentication password. | _none_ | +| `authorization_endpoint_url` | Authorization endpoint URL. | _none_ | +| `authorization_endpoint_username` | Authorization endpoint basic authentication username. | _none_ | +| `auto_retry_requests` | Immediately retry failed requests to AWS services once. This option doesn't affect the normal Fluent Bit retry mechanism with backoff. Instead, it enables an immediate retry with no delay for networking errors, which can help improve throughput during transient network issues. | `true` | +| `blob_database_file` | Absolute path to a database file to be used to store blob files contexts. | _none_ | +| `bucket` | S3 Bucket name | _none_ | +| `canned_acl` | [Predefined Canned ACL policy](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) for S3 objects. | _none_ | +| `compression` | Compression/format for S3 objects. Supported: `gzip` (always available) and `parquet` (requires Arrow build). For `gzip`, the `Content-Encoding` header is set to `gzip`. `parquet` is available **only when Fluent Bit is built with `-DFLB_ARROW=On`** and Arrow GLib/Parquet GLib are installed. Parquet is typically used with `use_put_object On`. | _none_ | +| `content_type` | A standard MIME type for the S3 object, set as the Content-Type HTTP header. | _none_ | +| `endpoint` | Custom endpoint for the S3 API. Endpoints can contain scheme and port. | _none_ | +| `external_id` | Specify an external ID for the STS API. Can be used with the `role_arn` parameter if your role requires an external ID. | _none_ | +| `file_delivery_attempt_limit` | File delivery attempt limit. | `1` | +| `host` | IP address or hostname of the target HTTP server. | `127.0.0.1` | +| `json_date_format` | Specify the format of the date. Accepted values: `double`, `epoch`, `epoch_ms`, `iso8601` (2018-05-30T09:39:52.000681Z), `_java_sql_timestamp_` (2018-05-30 09:39:52.000681). | _none_ | +| `json_date_key` | Specify the name of the date key in the output record. To disable the time key, set the value to `false`. | `date` | +| `log_level` | Specifies the log level for output plugin. If not set here, plugin uses global log level in `service` section. | `info` | +| `log_supress_interval` | Suppresses log messages from output plugin that appear similar within a specified time interval. `0` no suppression. | `0` | +| `match` | Set a tag pattern to match records that output should process. Exact matches or wildcards (for example `*`). | _none_ | +| `match_regex` | Set a regular expression to match tags for output routing. This allows more flexible matching compared to wildcards. | _none_ | +| `net.connect_timeout` | Set maximum time allowed to establish a connection, this time includes the TLS handshake. | `10s` | +| `net.connect_timeout_log_error` | On connection timeout, specify if it should log an error. When disabled, the timeout is logged as a debug message. | `true` | +| `net.keepalive_max_recycle` | Set maximum number of times a keepalive connection can be used before it retries. | `2000` | +| `net.dns.mode` | Select the primary DNS connection type (TCP or UDP). | _none_ | +| `net.dns.prefer_ipv4` | Select the primary DNS resolver type (LEGACY or ASYNC). | _none_ | +| `net.dns.prefer_ipv6` | Prioritize IPv6 DNS results when trying to establish a connection. | _none_ | +| `net.io_timeout` | Set maximum time a connection can stay idle while assigned. | `0s` | +| `net.max_worker_connections` | Set the maximum number of active TCP connections that can be used per worker thread. | `0` | +| `net.proxy_env_ignore` | Ignore the environment variables `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` when set. | `false` | +| `net.source_address` | Specify network address to bind for data traffic. | _none_ | +| `net.tcp_keepalive` | Enable or disable Keepalive support. | `off` | +| `net.tcp_keepalive_time` | Interval between the last data packet sent and the first TCP keepalive probe. | `-1` | +| `net.tcp_keepalive_interval` | Interval between TCP keepalive probes when no response is received on a `keepidle` probe. | `-1` | +| `net.tcp_keepalive_probes` | Number of unacknowledged probes to consider a connection dead. | `-1` | +| `log_key` | By default, the whole log record will be sent to S3. When specifying a key name with this option, only the value of that key sends to S3. For example, when using Docker you can specify `log_key log` and only the log message sends to S3. | _none_ | +| `part_delivery_attempt_limit` | File part delivery attempt limit. | `1` | +| `part_size` | Size of each part when uploading blob files. | `25M` | +| `port` | TCP port of the target HTTP server. | `80` | +| `preserve_data_ordering` | When an upload request fails, the last received chunk might swap with a later chunk, resulting in data shuffling. This feature prevents shuffling by using a queue logic for uploads. | `true` | +| `profile` | Option to specify an AWS Profile for credentials. | _none_ | +| `region` | The AWS region of your S3 bucket. | `us-east-1` | +| `retry_limit` | Set retry limit for output plugin when delivery fails. Integer, `no_limits`, `false`, or `off` to disable, or `no_retries` to disable retries entirely. | `1` | +| `role_arn` | ARN of an IAM role to assume (for example, for cross account access.) | _none_ | +| `s3_key_format` | Format string for keys in S3. This option supports a UUID, strftime time formatters, a syntax for selecting parts of the Fluent log tag using a syntax inspired by the `rewrite_tag` filter. Add `$UUID` in the format string to insert a random string. Add `$INDEX` in the format string to insert an integer that increments each upload. The `$INDEX` value saves in the `store_dir`. Add `$TAG` in the format string to insert the full log tag. Add `$TAG[0]` to insert the first part of the tag in the S3 key. The tag is split into parts using the characters specified with the `s3_key_format_tag_delimiters` option. Add the extension directly after the last piece of the format string to insert a key suffix. To specify a key suffix in `use_put_object` mode, you must specify `$UUID`. See [S3 Key Format](#s3-key-format-and-tag-delimiters). Time in `s3_key` is the timestamp of the first record in the S3 file. | `/fluent-bit-logs/$TAG/%Y/%m/%d/%H/%M/%S` | +| `s3_key_format_tag_delimiters` | A series of characters which will be used to split the tag into `parts` for use with the s3_key_format option. | `.` | +| `send_content_md5` | Send the Content-MD5 header with `PutObject` and UploadPart requests, as is required when Object Lock is enabled. | `false` | +| `static_file_path` | Disables behavior where UUID string appends to the end of the S3 key name when `$UUID` isn't provided in `s3_key_format`. `$UUID`, time formatters, `$TAG`, and other dynamic key formatters all work as expected while this feature is set to true. | `false` | +| `store_dir` | Directory to locally buffer data before sending. Plugin uses the S3 Multipart upload API to send data in chunks of 5 MB at a time. | `/tmp/fluent-bit/s3` | +| `store_dir_limit_size` | S3 plugin has its own buffering system with files in the `store_dir`. Use the `store_dir_limit_size` to limit the amount of data S3 buffers in the `store_dir` to limit disk usage. If the limit is reached, data will be discarded. Default is 0 which means unlimited. | `0` | +| `storage_class` | Specify the [storage class](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html#AmazonS3-PutObject-request-header-StorageClass) for S3 objects. If this option isn't specified, objects store with the default `STANDARD` storage class. | _none_ | +| `sts_endpoint` | Custom endpoint for the STS API. | _none_ | +| `tls` | Enable or disable TLS/SSL support. | `off` | +| `tls.ca_file` | Absolute path to CA certificate file. | _none_ | +| `tls.ca_path` | Absolute path to scan for certificate files. | _none_ | +| `tls.crt_file` | Absolute path to Certificate file. | _none_ | +| `tls.ciphers` | Specify TLS ciphers up to TLSv1.2. | _none_ | +| `tls.debug` | Set TLS debug level. Accepts `0` (No debug), `1`(Error), `2` (State change), `3` (Informational) and `4` (Verbose). | `1` | +| `tls.key_file` | Absolute path to private Key file. | _none_ | +| `tls.key_passwd` | Optional password for tls.key_file file. | _none_ | +| `tls.max_version` | Specify the maximum version of TLS. | _none_ | +| `tls.min_version` | Specify the minimum version of TLS. | _none_ | +| `tls.verify_hostname` | Enable or disable to verify hostname. | `off` | +| `tls.vhost` | Hostname to be used for TLS SNI extension. | _none_ | +| `tls.verify` | Force certificate validation. | `on` | +| `tls.windows.certstore_name` | Sets the `certstore` name on an output (Windows). | _none_ | +| `tls.windows.use_enterprise_store` | Sets whether using enterprise `certstore` or not on an output (Windows). | _none_ | +| `total_file_size` | Specify file size in S3. Minimum size is `1M`. With `use_put_object On` the maximum size is `1G`. With multipart uploads, the maximum size is `50G`. | `100000000` | +| `upload_chunk_size` | The size of each part for multipart uploads. Default: 5M, Max: 50M, Min: 5M. | `5242880` | +| `upload_part_freshness_timeout` | Maximum lifespan of an uncommitted file part. | `6D` | +| `upload_parts_timeout` | Timeout to upload parts of a blob file. | `10m` | +| `upload_timeout` | When this amount of time elapses, Fluent Bit uploads and creates a new file in S3. Set to `60m` to upload a new file every hour. | `10m` | +| `use_put_object` | Use the S3 `PutObject` API instead of the multipart upload API. When enabled, the key extension is only available when `$UUID` is specified in `s3_key_format`. If `$UUID` isn't included, a random string appends format string and the key extension can't be customized. | `false` | ## TLS / SSL @@ -389,7 +389,7 @@ With `use_put_object Off` (default), S3 will attempt to send files using multipa S3 [requires](https://docs.aws.amazon.com/AmazonS3/latest/userguide/qfacts.html) each [UploadPart](https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html) fragment to be at least 5,242,880 bytes, otherwise the upload is rejected. -The S3 output must sometimes fall back to the [`PutObject` API](https://docs.aws.amazon.com/AmazonS3/latest/API/API_`PutObject`.html). +The S3 output must sometimes fall back to the [`PutObject` API](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html). Uploads are triggered by these settings: @@ -420,7 +420,7 @@ On shutdown, S3 output attempts to complete all pending uploads. If an upload fa ## Usage with MinIO -[MinIO](https://min.io/) is a high-performance, S3 compatible object storage, and you can build your app with S3 capability without S3. +[MinIO](https://www.min.io/) is a high-performance, S3 compatible object storage, and you can build your app with S3 capability without S3. The following example runs a MinIO server at `localhost:9000`, and creates a bucket of `your-bucket`. diff --git a/pipeline/outputs/slack.md b/pipeline/outputs/slack.md index cc7903a62..98c11121a 100644 --- a/pipeline/outputs/slack.md +++ b/pipeline/outputs/slack.md @@ -2,11 +2,11 @@ The Slack output plugin delivers records or messages to your preferred Slack channel. It formats the outgoing content in JSON format for readability. -This connector uses Slack [incoming webhooks](https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks) to post messages to Slack channels. Using this plugin in conjunction with the Stream Processor is a good combination for alerting. +This connector uses Slack [incoming webhooks](https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks/) to post messages to Slack channels. Using this plugin in conjunction with the Stream Processor is a good combination for alerting. ## Slack webhook -Before configuring this plugin, set up your incoming webhook. For help, see the [Slack documentation](https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks). +Before configuring this plugin, set up your incoming webhook. For help, see the [Slack documentation](https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks/). After you have obtained the Webhook address you can place it in the configuration. @@ -14,8 +14,8 @@ After you have obtained the Webhook address you can place it in the configuratio This plugin supports the following parameters: -| Key | Description | Default | -|:--------|:------------|:--------| +| Key | Description | Default | +| --- | ----------- | ------- | | `webhook` | Absolute address of the webhook provided by Slack. | _none_ | | `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | diff --git a/pipeline/outputs/stackdriver.md b/pipeline/outputs/stackdriver.md index 1f57ea05c..59feab63a 100644 --- a/pipeline/outputs/stackdriver.md +++ b/pipeline/outputs/stackdriver.md @@ -1,12 +1,12 @@ # Stackdriver -The _Stackdriver_ output plugin lets you ingest your records into the [Google Cloud Stackdriver Logging](https://cloud.google.com/logging/) service. +The _Stackdriver_ output plugin lets you ingest your records into the [Google Cloud Stackdriver Logging](https://cloud.google.com/logging) service. -Before getting started with the plugin configuration, be sure to obtain the [proper credentials](https://cloud.google.com/logging/docs/agent/logging/authorization#create-service-account) to get access to the service. For best results, use a common JSON credentials file that can be referenced by the Stackdriver plugin. +Before getting started with the plugin configuration, be sure to obtain the [proper credentials](https://docs.cloud.google.com/logging/docs/agent/logging/authorization#create-service-account) to get access to the service. For best results, use a common JSON credentials file that can be referenced by the Stackdriver plugin. ## Configuration parameters -This plugin uses the following configuration parameters. For more details about Stackdriver-specific parameters, see the [Google Cloud `LogEntry` API documentation](https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry). +This plugin uses the following configuration parameters. For more details about Stackdriver-specific parameters, see the [Google Cloud `LogEntry` API documentation](https://docs.cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry). | Key | Description | Default | | --- | ----------- | ------- | @@ -216,7 +216,7 @@ This will produce the following log: This makes the `resource_labels` API the recommended choice for supporting new or existing resource types that have all resource labels known before runtime or available on the payload during runtime. -For instance, for a Kubernetes resource type, `resource_labels` can be used in tandem with the [Kubernetes filter](https://docs.fluentbit.io/manual/pipeline/filters/kubernetes) to pack all six resource labels. The following example shows what this could look like for a `k8s_container` resource: +For instance, for a Kubernetes resource type, `resource_labels` can be used in tandem with the [Kubernetes filter](../filters/kubernetes) to pack all six resource labels. The following example shows what this could look like for a `k8s_container` resource: {% tabs %} {% tab title="fluent-bit.yaml" %} @@ -255,7 +255,7 @@ By default, the plugin will write to the following log name: /projects//logs/ ``` -You might be in a scenario where being more specific about the log name is important (for example, [integration with Log Router rules](https://cloud.google.com/logging/docs/routing/overview) or [controlling cardinality of log based metrics](https://cloud.google.com/logging/docs/logs-based-metrics/troubleshooting#too-many-time-series)). You can control the log name directly on a per-log basis by using the [`logging.googleapis.com/logName` special field](https://github.com/fluent/fluent-bit-docs/blob/master/pipeline/outputs/stackdriver_special_fields.md#log-entry-fields). You can configure a `log_name_key` if you'd like to use something different from `logging.googleapis.com/logName`. For example, if the `log_name_key` is set to `mylognamefield` will extract the log name from `mylognamefield` in the log. +You might be in a scenario where being more specific about the log name is important (for example, [integration with Log Router rules](https://docs.cloud.google.com/logging/docs/routing/overview) or [controlling cardinality of log based metrics](https://cloud.google.com/logging/docs/logs-based-metrics/troubleshooting#too-many-time-series)). You can control the log name directly on a per-log basis by using the [`logging.googleapis.com/logName` special field](https://github.com/fluent/fluent-bit-docs/blob/master/pipeline/outputs/stackdriver_special_fields.md#log-entry-fields). You can configure a `log_name_key` if you'd like to use something different from `logging.googleapis.com/logName`. For example, if the `log_name_key` is set to `mylognamefield` will extract the log name from `mylognamefield` in the log. ## Troubleshooting @@ -269,8 +269,8 @@ An upstream connection error means Fluent Bit wasn't able to reach Google servic This is due to a network issue in the environment where Fluent Bit is running. Make sure that the Host, Container or Pod can reach the following Google endpoints: -- [https://www.googleapis.com](https://www.googleapis.com) -- [https://logging.googleapis.com](https://logging.googleapis.com) +- [https://www.googleapis.com](https://cloud.google.com/apis) +- [https://logging.googleapis.com](https://docs.cloud.google.com/logging/docs/apis) {% hint style="warning" %} @@ -293,6 +293,6 @@ Check the following: ## Other Implementations -Stackdriver officially supports a [logging agent based on Fluentd](https://cloud.google.com/logging/docs/agent). +Stackdriver officially supports a [logging agent based on Fluentd](https://docs.cloud.google.com/logging/docs/agent). -Fluent Bit plans to support some [special fields in structured payloads](https://cloud.google.com/logging/docs/agent/configuration#special-fields). For more information, see the documentation about [Stackdriver Special Fields](stackdriver_special_fields.md#logentry-fields). +Fluent Bit plans to support some [special fields in structured payloads](https://docs.cloud.google.com/logging/docs/agent/configuration#special-fields). For more information, see the documentation about [Stackdriver Special Fields](stackdriver_special_fields.md#logentry-fields). From 8ce44653d04de6be31206a1a523d8e5928971531 Mon Sep 17 00:00:00 2001 From: Lynette Miles Date: Tue, 3 Mar 2026 09:46:19 -0800 Subject: [PATCH 2/4] I think this is most of the rest of the redirects Signed-off-by: Lynette Miles --- README.md | 2 +- .../outputs/stackdriver_special_fields.md | 20 +++++++++---------- pipeline/outputs/tcp-and-tls.md | 12 +++++------ pipeline/outputs/treasure-data.md | 6 +++--- pipeline/outputs/vivo-exporter.md | 2 +- pipeline/parsers/configuring-parser.md | 2 +- pipeline/parsers/regular-expression.md | 2 +- 7 files changed, 23 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index 430661687..7927a15c9 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@ For more details about changes in each release, refer to the [official release n ## Fluent Bit, Fluentd, and CNCF -Fluent Bit is a [CNCF](https://cncf.io) graduated sub-project under the umbrella of [Fluentd](https://www.fluentd.org). +Fluent Bit is a [CNCF](https://www.cncf.io/) graduated sub-project under the umbrella of [Fluentd](https://www.fluentd.org). Fluent Bit was originally created by [Eduardo Silva](https://www.linkedin.com/in/edsiper/) and is now sponsored by [Chronosphere](https://chronosphere.io/). As a CNCF-hosted project, it's a fully vendor-neutral and community-driven project. diff --git a/pipeline/outputs/stackdriver_special_fields.md b/pipeline/outputs/stackdriver_special_fields.md index 8d2fdcb3d..f0b118548 100644 --- a/pipeline/outputs/stackdriver_special_fields.md +++ b/pipeline/outputs/stackdriver_special_fields.md @@ -1,21 +1,21 @@ # Stackdriver special fields -When the [google-logging-agent](https://cloud.google.com/logging/docs/agent) receives a structured log record, it treats [some fields](https://cloud.google.com/logging/docs/agent/configuration#special-fields) specially, allowing users to set specific fields in the LogEntry object that get written to the Logging API. +When the [google-logging-agent](https://docs.cloud.google.com/logging/docs/agent) receives a structured log record, it treats [some fields](https://docs.cloud.google.com/logging/docs/agent/logging/configuration#special-fields) specially, allowing users to set specific fields in the LogEntry object that get written to the Logging API. ## LogEntry fields Fluent Bit support some special fields for setting fields on the LogEntry object: -| JSON log field | [LogEntry](https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry) field | Type | Description | +| JSON log field | [LogEntry](https://docs.cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry) field | Type | Description | | :--- | :--- | :--- | :--- | | `logging.googleapis.com/logName` | `logName` | `string` | The log name to write this log to. | | `logging.googleapis.com/labels` | `labels` | `object` | The labels for this log. | -| `logging.googleapis.com/severity` | `severity` | [`LogSeverity` Enum](https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogSeverity) | The severity of this log. | -| `logging.googleapis.com/monitored_resource` | `resource` | [`MonitoredResource`](https://cloud.google.com/logging/docs/reference/v2/rest/v2/MonitoredResource) (without `type`) | Resource labels for this log. | -| `logging.googleapis.com/operation` | `operation` | [`LogEntryOperation`](https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogEntryOperation) | Additional information about a potentially long-running operation. | +| `logging.googleapis.com/severity` | `severity` | [`LogSeverity` Enum](https://docs.cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogSeverity) | The severity of this log. | +| `logging.googleapis.com/monitored_resource` | `resource` | [`MonitoredResource`](https://docs.cloud.google.com/logging/docs/reference/v2/rest/v2/MonitoredResource) (without `type`) | Resource labels for this log. | +| `logging.googleapis.com/operation` | `operation` | [`LogEntryOperation`](https://docs.cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogEntryOperation) | Additional information about a potentially long-running operation. | | `logging.googleapis.com/insertId` | `insertId` | `string` | A unique identifier for the log entry. It's used to order `logEntries`. | -| `logging.googleapis.com/sourceLocation` | `sourceLocation` | [`LogEntrySourceLocation`](https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogEntrySourceLocation) | Additional information about the source code location that produced the log entry. | -| `logging.googleapis.com/http_request` | `httpRequest` | [`HttpRequest`](https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#HttpRequest) | A common proto for logging HTTP requests. | +| `logging.googleapis.com/sourceLocation` | `sourceLocation` | [`LogEntrySourceLocation`](https://docs.cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogEntrySourceLocation) | Additional information about the source code location that produced the log entry. | +| `logging.googleapis.com/http_request` | `httpRequest` | [`HttpRequest`](https://docs.cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#HttpRequest) | A common proto for logging HTTP requests. | | `logging.googleapis.com/trace` | `trace` | `string` | Resource name of the trace associated with the log entry. | | `logging.googleapis.com/traceSampled` | `traceSampled` | boolean | The sampling decision associated with this log entry. | | `logging.googleapis.com/spanId` | `spanId` | `string` | The ID of the trace span associated with this log entry. | @@ -25,9 +25,9 @@ Fluent Bit support some special fields for setting fields on the LogEntry object ## Other special fields | JSON log field | Description | -|:---------------|:------------| +| -------------- | ----------- | | `logging.googleapis.com/projectId` | Changes the project ID that this log will be written to. Ensure that you are authenticated to write logs to this project. | -| `logging.googleapis.com/local_resource_id` | Overrides the [configured `local_resource_id`](./stackdriver.md#resource-labels). | +| `logging.googleapis.com/local_resource_id` | Overrides the [configured `local_resource_id`](./stackdriver.md#resource-labels). | ## Use special fields @@ -62,7 +62,7 @@ For the special fields that map to `LogEntry` prototypes, add them as objects wi } ``` -Adding special fields to logs is best done through the [`modify` filter](https://docs.fluentbit.io/manual/pipeline/filters/modify) for basic fields, or [a Lua script using the `lua` filter](https://docs.fluentbit.io/manual/pipeline/filters/lua) for more complex fields. +Adding special fields to logs is best done through the [`modify` filter](../filters/modify) for basic fields, or [a Lua script using the `lua` filter](../filters/lua) for more complex fields. ## Basic type special fields diff --git a/pipeline/outputs/tcp-and-tls.md b/pipeline/outputs/tcp-and-tls.md index 8a3f44f34..03c3add17 100644 --- a/pipeline/outputs/tcp-and-tls.md +++ b/pipeline/outputs/tcp-and-tls.md @@ -7,11 +7,11 @@ The _TCP and TLS_ output plugin lets you send records to a remote TCP server. Th This plugin supports the following parameters: | Key | Description | Default | -|:--- |:----------- |:------- | +| --- | ----------- | ------- | | `Host` | Target host where Fluent Bit or Fluentd are listening for Forward messages. | `127.0.0.1` | | `Port` | TCP Port of the target service. | `5170` | | `Format` | Specify the data format to be printed. Supported formats: `msgpack`, `json`, `json_lines`, `json_stream`. | `msgpack` | -| `json_date_key`| Specify the name of the time key in the output record. To disable the time key, set the value to `false`. | `date` | +| `json_date_key` | Specify the name of the time key in the output record. To disable the time key, set the value to `false`. | `date` | | `json_date_format` | Specify the format of the date. Supported formats: `double`, `epoch`, `epoch_ms`, `iso8601`, `java_sql_timestamp`. | `double` | | `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `2` | @@ -20,20 +20,20 @@ This plugin supports the following parameters: The following parameters are available to configure a secure channel connection through TLS: | Key | Description | Default | -|:--- |:----------- |:------- | +| --- | ----------- | ------- | | `tls` | Enable or disable TLS support. | `Off` | | `tls.verify` | Force certificate validation. | `On` | | `tls.debug` | Set TLS debug verbosity level. Allowed values: `0` (No debug), `1` (Error), `2` (State change), `3` (Informational), `4` (Verbose) | `1` | | `tls.ca_file` | Absolute path to CA certificate file. | _none_ | -| `tls.crt_file` | Absolute path to Certificate file. | _none_ | -| `tls.key_file` | Absolute path to private Key file. | _none_ | +| `tls.crt_file` | Absolute path to Certificate file. | _none_ | +| `tls.key_file` | Absolute path to private Key file. | _none_ | | `tls.key_passwd` | Optional password for `tls.key_file` file. | _none_ | ### Command line #### JSON format -This example specifies gathering [CPU](https://docs.fluentbit.io/manual/pipeline/inputs/cpu-metrics) usage metrics and send them in JSON lines mode to a remote endpoint using netcat service. +This example specifies gathering [CPU](../inputs/cpu-metrics) usage metrics and send them in JSON lines mode to a remote endpoint using netcat service. ```shell fluent-bit -i cpu -o tcp://127.0.0.1:5170 -p format=json_lines -v diff --git a/pipeline/outputs/treasure-data.md b/pipeline/outputs/treasure-data.md index c017942f0..f2b0cb508 100644 --- a/pipeline/outputs/treasure-data.md +++ b/pipeline/outputs/treasure-data.md @@ -7,12 +7,12 @@ The _Treasure Data_ (TD) output plugin lets you flush your records into the [Tre The plugin supports the following configuration parameters: | Key | Description | Default | -|:--- |:----------- |:--------| -| `API` | The Treasure Data API key. To obtain it, log into the [Console](https://console.treasuredata.com) and in the API keys box, copy the API key hash. | _none_ | +| --- | ----------- | ------- | +| `API` | The Treasure Data API key. To obtain it, log into the [Console](https://console.us01.treasuredata.com/users/sign_in) and in the API keys box, copy the API key hash. | _none_ | | `Database` | Specify the name of your target database. | _none_ | | `Table` | Specify the name of your target table where the records will be stored. | _none_ | | `Region` | Set the service region. Allowed values: `US`, `JP`. | `US` | -| `Workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | +| `Workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | ## Get started diff --git a/pipeline/outputs/vivo-exporter.md b/pipeline/outputs/vivo-exporter.md index 24618202e..6db0cead5 100644 --- a/pipeline/outputs/vivo-exporter.md +++ b/pipeline/outputs/vivo-exporter.md @@ -1,6 +1,6 @@ # Vivo Exporter -Vivo Exporter is an output plugin that exposes logs, metrics, and traces through an HTTP endpoint. This plugin aims to be used in conjunction with [Vivo project](https://github.com/calyptia/vivo) . +Vivo Exporter is an output plugin that exposes logs, metrics, and traces through an HTTP endpoint. This plugin aims to be used in conjunction with [Vivo project](https://github.com/chronosphereio/calyptia-vivo) . ## Configuration parameters diff --git a/pipeline/parsers/configuring-parser.md b/pipeline/parsers/configuring-parser.md index 297d2f335..a05d68435 100644 --- a/pipeline/parsers/configuring-parser.md +++ b/pipeline/parsers/configuring-parser.md @@ -26,7 +26,7 @@ Custom parsers support the following configuration parameters: | `time_key` | If the log entry provides a field with a timestamp, this option specifies the name of that field. | _none_ | | `time_offset` | Specifies a fixed UTC time offset (such as `-0600` or `+0200`) for local dates. | _none_ | | `time_strict` | If `true`, the parser is strict with the expected time format. If `false`, the parser is permissive with the format of the time. Set to `false` when the format expects a time fraction but the time to be parsed doesn't include it. | `true` | -| `time_system_timezone` | If there is no time zone (`%z`) specified in the given `time_format`, enabling this option will make the parser detect and use the system's configured time zone. The configured time zone is detected from the [`TZ` environment variable](https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html). | `false` | +| `time_system_timezone` | If there is no time zone (`%z`) specified in the given `time_format`, enabling this option will make the parser detect and use the system's configured time zone. The configured time zone is detected from the [`TZ` environment variable](https://sourceware.org/glibc/manual/latest/html_node/TZ-Variable.html). | `false` | | `types` | Specifies the data type of a parsed field. The syntax is `types : : ...`. The supported types are `string` (default), `integer`, `bool`, `float`, `hex`. | _none_ | ### Time resolution and fractional seconds diff --git a/pipeline/parsers/regular-expression.md b/pipeline/parsers/regular-expression.md index a8a40c5e8..308c2a16f 100644 --- a/pipeline/parsers/regular-expression.md +++ b/pipeline/parsers/regular-expression.md @@ -24,7 +24,7 @@ The `regex` parser supports the following format-specific configuration paramete Fluent Bit uses the [Onigmo](https://github.com/k-takata/Onigmo) regular expression library in Ruby mode. -You can use only alphanumeric characters and underscore in group names. For example, a group name like `(?.*)` causes an error due to the invalid dash (`-`) character. Use the [Rubular](http://rubular.com/) web editor to test your expressions. +You can use only alphanumeric characters and underscore in group names. For example, a group name like `(?.*)` causes an error due to the invalid dash (`-`) character. Use the [Rubular](https://rubular.com/) web editor to test your expressions. The following parser configuration example provides rules that can be applied to an Apache HTTP Server log entry: From fc1867c0faf0b275802aa9f74183955aa4d953d4 Mon Sep 17 00:00:00 2001 From: Lynette Miles <6818907+esmerel@users.noreply.github.com> Date: Tue, 3 Mar 2026 10:11:36 -0800 Subject: [PATCH 3/4] Apply suggestions from code review Signed-off-by: Lynette Miles <6818907+esmerel@users.noreply.github.com> --- pipeline/outputs/treasure-data.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/outputs/treasure-data.md b/pipeline/outputs/treasure-data.md index f2b0cb508..bec6bbbf2 100644 --- a/pipeline/outputs/treasure-data.md +++ b/pipeline/outputs/treasure-data.md @@ -8,7 +8,7 @@ The plugin supports the following configuration parameters: | Key | Description | Default | | --- | ----------- | ------- | -| `API` | The Treasure Data API key. To obtain it, log into the [Console](https://console.us01.treasuredata.com/users/sign_in) and in the API keys box, copy the API key hash. | _none_ | +| `API` | The Treasure Data API key. To obtain it, log into the [Console](https://console.treasuredata.com/users/sign_in) and in the API keys box, copy the API key hash. | _none_ | | `Database` | Specify the name of your target database. | _none_ | | `Table` | Specify the name of your target table where the records will be stored. | _none_ | | `Region` | Set the service region. Allowed values: `US`, `JP`. | `US` | From da1bec4f0b834d1767e949a3c337dbe9a22a65e4 Mon Sep 17 00:00:00 2001 From: Lynette Miles <6818907+esmerel@users.noreply.github.com> Date: Tue, 3 Mar 2026 10:12:17 -0800 Subject: [PATCH 4/4] Apply suggestions from code review Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Signed-off-by: Lynette Miles <6818907+esmerel@users.noreply.github.com> --- pipeline/outputs/vivo-exporter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/outputs/vivo-exporter.md b/pipeline/outputs/vivo-exporter.md index 6db0cead5..a530206c9 100644 --- a/pipeline/outputs/vivo-exporter.md +++ b/pipeline/outputs/vivo-exporter.md @@ -1,6 +1,6 @@ # Vivo Exporter -Vivo Exporter is an output plugin that exposes logs, metrics, and traces through an HTTP endpoint. This plugin aims to be used in conjunction with [Vivo project](https://github.com/chronosphereio/calyptia-vivo) . +Vivo Exporter is an output plugin that exposes logs, metrics, and traces through an HTTP endpoint. This plugin aims to be used in conjunction with [Vivo project](https://github.com/chronosphereio/calyptia-vivo). ## Configuration parameters