Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ This repository runs linters as GitHub Actions for each pull request. If a linte

### Vale

[Vale](https://vale.sh/docs/) lints prose for style and clarity. In addition to reviewing the results of each Vale test in GitHub, you can use the [Vale plugin for VSCode](https://marketplace.visualstudio.com/items?itemName=ChrisChinchilla.vale-vscode) to view errors and suggestions locally.
[Vale](https://vale.sh/docs) lints prose for style and clarity. In addition to reviewing the results of each Vale test in GitHub, you can use the [Vale plugin for VSCode](https://marketplace.visualstudio.com/items?itemName=ChrisChinchilla.vale-vscode) to view errors and suggestions locally.

Vale tests for the Fluent Bit docs are stored in the [`/vale-styles`](https://github.com/fluent/fluent-bit-docs/tree/master/vale-styles) folder. Most Vale tests are at the `suggestion` or `warning` level and won't block pull requests from merging. However, tests at the `error` level will block merging until the associated issue is fixed.

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ For more details about changes in each release, refer to the [official release n

## Fluent Bit, Fluentd, and CNCF

Fluent Bit is a [CNCF](https://cncf.io) graduated sub-project under the umbrella of [Fluentd](https://www.fluentd.org).
Fluent Bit is a [CNCF](https://www.cncf.io/) graduated sub-project under the umbrella of [Fluentd](https://www.fluentd.org).

Fluent Bit was originally created by [Eduardo Silva](https://www.linkedin.com/in/edsiper/) and is now sponsored by [Chronosphere](https://chronosphere.io/). As a CNCF-hosted project, it's a fully vendor-neutral and community-driven project.

Expand Down
2 changes: 1 addition & 1 deletion about/fluentd-and-fluent-bit.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Telemetry data processing can be complex, especially at scale. That's why [Fluen
The Fluentd and Fluent Bit projects are both:

- Licensed under the terms of Apache License v2.0.
- Graduated hosted projects by the [Cloud Native Computing Foundation (CNCF)](https://cncf.io).
- Graduated hosted projects by the [Cloud Native Computing Foundation (CNCF)](https://www.cncf.io/).
- Production-grade solutions, with Fluent Bit deployed over 15 billion times globally.
- Vendor neutral and community driven.
- Widely adopted by the industry, being trusted by major companies like Amazon, Microsoft, Google, and hundreds of others.
Expand Down
2 changes: 1 addition & 1 deletion administration/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -469,7 +469,7 @@ When querying the related metrics, the aliases are returned instead of the plugi

You can create Grafana dashboards and alerts using Fluent Bit exposed Prometheus style metrics.

The provided [example dashboard](https://github.com/fluent/fluent-bit-docs/blob/master/monitoring/dashboard.json) is heavily inspired by [Banzai Cloud](https://github.com/banzaicloud)'s [logging operator dashboard](https://grafana.com/grafana/dashboards/7752-logging-dashboard) with a few key differences, such as the use of the `instance` label, stacked graphs, and a focus on Fluent Bit metrics. See [this blog post](https://www.robustperception.io/controlling-the-instance-label/) for more information.
The provided [example dashboard](https://github.com/fluent/fluent-bit-docs/blob/master/monitoring/dashboard.json) is heavily inspired by [Banzai Cloud](https://github.com/banzaicloud)'s [logging operator dashboard](https://grafana.com/grafana/dashboards/7752-logging-dashboard/) with a few key differences, such as the use of the `instance` label, stacked graphs, and a focus on Fluent Bit metrics. See [this blog post](https://www.robustperception.io/controlling-the-instance-label/) for more information.

![dashboard](/.gitbook/assets/dashboard.png)

Expand Down
2 changes: 1 addition & 1 deletion installation/downloads.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,4 +50,4 @@

## Enterprise providers

Fluent Bit packages are also provided by [enterprise providers](https://fluentbit.io/enterprise) for older end-of-life versions, Unix systems, or for additional support and features including aspects (such as CVE backporting).
Fluent Bit packages are also provided by [enterprise providers](https://fluentbit.io/enterprise/) for older end-of-life versions, Unix systems, or for additional support and features including aspects (such as CVE backporting).

Check warning on line 53 in installation/downloads.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Acronyms] Spell out 'CVE', if it's unfamiliar to the audience. Raw Output: {"message": "[FluentBit.Acronyms] Spell out 'CVE', if it's unfamiliar to the audience.", "location": {"path": "installation/downloads.md", "range": {"start": {"line": 53, "column": 211}}}, "severity": "INFO"}
2 changes: 1 addition & 1 deletion pipeline/filters/aws-metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ To achieve this, AWS Filter can be configured with `tags_enabled true` to enable

### Requirements

To use the `tags_enabled true` feature in Fluent Bit, the [instance-metadata-tags](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/modify-instance-metadata-options.html) option must be enabled on the EC2 instance where Fluent Bit is running. Without this option enabled, Fluent Bit won't be able to retrieve the tags associated with the EC2 instance. However, this doesn't mean that Fluent Bit will fail or stop working altogether. Instead, if [instance-metadata-tags](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/modify-instance-metadata-options.html) option isn't enabled, Fluent Bit will continue to operate normally and capture other values, such as the EC2 instance ID or availability zone, based on its configuration.
To use the `tags_enabled true` feature in Fluent Bit, the [`instance-metadata-tags`](https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-metadata-options.html) option must be enabled on the EC2 instance where Fluent Bit is running. Without this option enabled, Fluent Bit won't be able to retrieve the tags associated with the EC2 instance. However, this doesn't mean that Fluent Bit will fail or stop working altogether. Instead, if the `instance-metadata-tags` option isn't enabled, Fluent Bit will continue to operate normally and capture other values, such as the EC2 instance ID or availability zone, based on its configuration.

### Example

Expand Down
2 changes: 1 addition & 1 deletion pipeline/filters/kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -680,4 +680,4 @@ Learn how to solve them to ensure that the Fluent Bit Kubernetes filter is opera

## Credit

The Kubernetes Filter plugin is fully inspired by the [Fluentd Kubernetes Metadata Filter](https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter) written by [`Jimmi Dyson`](https://github.com/jimmidyson).
The Kubernetes Filter plugin is fully inspired by the [Fluentd Kubernetes Metadata Filter](https://github.com/fluent-plugins-nursery/fluent-plugin-kubernetes_metadata_filter) written by [`Jimmi Dyson`](https://github.com/jimmidyson).
2 changes: 1 addition & 1 deletion pipeline/inputs/cpu-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ The command returns results similar to the following:
...
```

As described previously, the CPU input plugin gathers the overall usage every one second and flushed the information to the output on the fifth second. This example uses the `stdout` plugin to demonstrate the output records. In a real use-case you might want to flush this information to some central aggregator such as [Fluentd](https://fluentd.org) or [Elasticsearch](https://elastic.co).
As described previously, the CPU input plugin gathers the overall usage every one second and flushed the information to the output on the fifth second. This example uses the `stdout` plugin to demonstrate the output records. In a real use-case you might want to flush this information to some central aggregator such as [Fluentd](https://www.fluentd.org/) or [Elasticsearch](https://www.elastic.co/).

### Configuration file

Expand Down
2 changes: 1 addition & 1 deletion pipeline/inputs/kafka.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ The _Kafka_ input plugin enables Fluent Bit to consume messages directly from on

Starting with version 4.0.4, the Kafka input plugin supports authentication with AWS MSK IAM, enabling integration with Amazon MSK (Managed Streaming for Apache Kafka) clusters that require IAM-based access.

This plugin uses the official [librdkafka C library](https://github.com/edenhill/librdkafka) as a built-in dependency.
This plugin uses the official [librdkafka C library](https://github.com/confluentinc/librdkafka) as a built-in dependency.

## Configuration parameters

Expand Down
2 changes: 1 addition & 1 deletion pipeline/inputs/windows-event-log-winevtlog.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ The default value of `Read_Limit_Per_Cycle` is `512KiB`.
#### Query languages for `Event_Query` parameter

The `Event_Query` parameter can be used to specify the XML query for filtering Windows EventLog during collection.
The supported query types are [`XPath`](https://developer.mozilla.org/en-US/docs/Web/XPath) and XML Query.
The supported query types are [`XPath`](https://developer.mozilla.org/en-US/docs/Web/XML/XPath) and XML Query.
For further details, refer to [Microsoft's documentation](https://learn.microsoft.com/en-us/windows/win32/wes/consuming-events).

### Command line
Expand Down
20 changes: 10 additions & 10 deletions pipeline/outputs/bigquery.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
# Google Cloud BigQuery

The _Google Cloud BigQuery_ output plugin is an experimental plugin that lets you stream records into the [Google Cloud BigQuery](https://cloud.google.com/bigquery/) service.
The _Google Cloud BigQuery_ output plugin is an experimental plugin that lets you stream records into the [Google Cloud BigQuery](https://cloud.google.com/bigquery) service.

The implementation doesn't support the following, which would be expected in a full production version:

- [Application Default Credentials](https://cloud.google.com/docs/authentication/production).
- [Data deduplication](https://cloud.google.com/bigquery/streaming-data-into-bigquery) using `insertId`.
- [Template tables](https://cloud.google.com/bigquery/streaming-data-into-bigquery) using `templateSuffix`.
- [Application Default Credentials](https://docs.cloud.google.com/docs/authentication).
- [Data deduplication](https://docs.cloud.google.com/bigquery/docs/streaming-data-into-bigquery) using `insertId`.
- [Template tables](https://docs.cloud.google.com/bigquery/docs/streaming-data-into-bigquery) using `templateSuffix`.

## Google Cloud configuration

Fluent Bit streams data into an existing BigQuery table using a service account that you specify. Before using the BigQuery output plugin, you must:

1. To stream data into BigQuery, you must create a [Google Cloud service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts) for Fluent Bit.
1. To stream data into BigQuery, you must create a [Google Cloud service account](https://docs.cloud.google.com/iam/docs/service-accounts-create) for Fluent Bit.
1. Create a BigQuery dataset.
Fluent Bit doesn't create datasets for your data, so you must [create the dataset](https://cloud.google.com/bigquery/docs/datasets) ahead of time. You must also grant the service account `WRITER` permission on the dataset.
Fluent Bit doesn't create datasets for your data, so you must [create the dataset](https://docs.cloud.google.com/bigquery/docs/datasets) ahead of time. You must also grant the service account `WRITER` permission on the dataset.

Within the dataset you must create a table for the data to reside in. Use the following instructions for creating your table. Pay close attention to the schema, as it must match the schema of your output JSON. Unfortunately, because BigQuery doesn't allow dots in field names, you must use a filter to change the fields for many of the standard inputs (for example, `mem` or `cpu`).
1. [Create a BigQuery table](https://cloud.google.com/bigquery/docs/tables).
1. Fluent Bit BigQuery output plugin uses a JSON credentials file for authentication credentials. [Authorize the service account](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) to write to the table.
1. [Create a BigQuery table](https://docs.cloud.google.com/bigquery/docs/tables).
1. Fluent Bit BigQuery output plugin uses a JSON credentials file for authentication credentials. [Authorize the service account](https://docs.cloud.google.com/iam/docs/keys-create-delete) to write to the table.
1. Provide the service account credentials to Fluent Bit.
With [workload identity federation](https://cloud.google.com/iam/docs/workload-identity-federation), you can grant on-premises or multi-cloud workloads access to Google Cloud resources, without using a service account key. It can be used as a more secure alternative to service account credentials. Google Cloud's workload identity federation supports several identity providers (see documentation) but Fluent Bit BigQuery plugin currently supports Amazon Web Services (AWS) only.
With [workload identity federation](https://docs.cloud.google.com/iam/docs/workload-identity-federation), you can grant on-premises or multi-cloud workloads access to Google Cloud resources, without using a service account key. It can be used as a more secure alternative to service account credentials. Google Cloud's workload identity federation supports several identity providers (see documentation) but Fluent Bit BigQuery plugin currently supports Amazon Web Services (AWS) only.

You must configure workload identity federation in GCP before using it with Fluent Bit.

Expand All @@ -45,7 +45,7 @@ Fluent Bit streams data into an existing BigQuery table using a service account
| `google_service_account` | The email address of the Google service account to impersonate. The workload identity provider must have permissions to impersonate this service account, and the service account must have permissions to access Google BigQuery resources ( `write` access to tables) | _none_ |
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |

See Google's [official documentation](https://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll) for further details.
See Google's [official documentation](https://docs.cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll) for further details.

## Configuration file

Expand Down
8 changes: 4 additions & 4 deletions pipeline/outputs/chronicle.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
# Google Chronicle

The _Google Chronicle_ output plugin lets you ingest security logs into the [Google Chronicle](https://chronicle.security/) service. This connector is designed to send unstructured security logs.
The _Google Chronicle_ output plugin lets you ingest security logs into the [Google Chronicle](https://cloud.google.com/security/products/security-operations) service. This connector is designed to send unstructured security logs.

## Google Cloud configuration

Fluent Bit streams data into an existing Google Chronicle tenant using a service account that you specify. Before using the Chronicle output plugin, you must:

1. Create a service account.

To stream security logs into Google Chronicle, create a [Google Cloud service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts) for Fluent Bit:
To stream security logs into Google Chronicle, create a [Google Cloud service account](https://docs.cloud.google.com/iam/docs/service-accounts-create) for Fluent Bit:

1. Create a tenant of Google Chronicle.

Fluent Bit doesn't create a tenant of Google Chronicle for your security logs, so you must create this ahead of time.

1. Retrieve service account credentials.

The Fluent Bit Chronicle output plugin uses a JSON credentials file for authentication credentials. Download the credentials file by following the instructions for [Creating and Managing Service Account Keys](https://cloud.google.com/iam/docs/creating-managing-service-account-keys).
The Fluent Bit Chronicle output plugin uses a JSON credentials file for authentication credentials. Download the credentials file by following the instructions for [Creating and Managing Service Account Keys](https://docs.cloud.google.com/iam/docs/keys-create-delete).

## Configurations parameters

Expand All @@ -27,7 +27,7 @@ Fluent Bit streams data into an existing Google Chronicle tenant using a service
| `service_account_secret` | Private key content associated with the service account. Only available if no credentials file has been provided. | Value of environment variable `$SERVICE_ACCOUNT_SECRET` |
| `project_id` | The project id containing the tenant of Google Chronicle to stream into. | The value of the `project_id` in the credentials file |
| `customer_id` | The customer id to identify the tenant of Google Chronicle to stream into. The value of the `customer_id` should be specified in the configuration file. | _none_ |
| `log_type` | The log type to parse logs as. Google Chronicle supports parsing for [specific log types only](https://cloud.google.com/chronicle/docs/ingestion/parser-list/supported-default-parsers). | _none_ |
| `log_type` | The log type to parse logs as. Google Chronicle supports parsing for [specific log types only](https://docs.cloud.google.com/chronicle/docs/ingestion/parser-list/supported-default-parsers). | _none_ |
| `region` | The GCP region in which to store security logs. Supported regions: `US`, `EU`, `UK`, `ASIA`. Blank is handled as `US`. | _none_ |
| `log_key` | By default, the whole log record will be sent to Google Chronicle. If you specify a key name with this option, then only the value of that key will be sent to Google Chronicle. | _none_ |
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |
Expand Down
2 changes: 1 addition & 1 deletion pipeline/outputs/cloudwatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -407,7 +407,7 @@ If you see errors for image pull limits, try log into public ECR with your AWS c
aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws
```

You can check the [Amazon ECR Public official doc](https://docs.aws.amazon.com/AmazonECR/latest/public/get-set-up-for-amazon-ecr.html) for more details
You can check the [Amazon ECR Public official doc](https://docs.aws.amazon.com/AmazonECR/latest/public/what-is-ecr.html) for more details

#### Docker Hub

Expand Down
Loading
Loading