diff --git a/administration/configuring-fluent-bit.md b/administration/configuring-fluent-bit.md index 01a17db98..150b0059b 100644 --- a/administration/configuring-fluent-bit.md +++ b/administration/configuring-fluent-bit.md @@ -1,6 +1,6 @@ # Configure Fluent Bit -Fluent Bit uses configuration files to store information about your specified [inputs](../data-pipeline/inputs.md), [outputs](../data-pipeline/outputs.md), [filters](../data-pipeline/filters.md), and more. You can write these configuration files in one of these formats: +Fluent Bit uses configuration files to store information about your specified [inputs](../pipeline/inputs.md), [outputs](../pipeline/outputs.md), [filters](../pipeline/filters.md), and more. You can write these configuration files in one of these formats: - [YAML configuration files](./configuring-fluent-bit/yaml.md) are the standard configuration format as of Fluent Bit v3.2. They use the `.yaml` file extension. - [Classic configuration files](./configuring-fluent-bit/classic-mode.md) will be deprecated at the end of 2026. They use the `.conf` file extension. diff --git a/administration/configuring-fluent-bit/yaml.md b/administration/configuring-fluent-bit/yaml.md index 19998f906..f0edf6957 100644 --- a/administration/configuring-fluent-bit/yaml.md +++ b/administration/configuring-fluent-bit/yaml.md @@ -3,19 +3,19 @@ In Fluent Bit v3.2 and later, YAML configuration files support all of the settings -and features that [classic configuration files](../administration/configuring-fluent-bit/classic-mode.md) support, plus additional features that classic configuration files +and features that [classic configuration files](./classic-mode.md) support, plus additional features that classic configuration files don't support, like processors. YAML configuration files support the following top-level sections: -- `env`: Configures [environment variables](../administration/configuring-fluent-bit/yaml/environment-variables-section). -- `includes`: Specifies additional YAML configuration files to [include as part of a parent file](../administration/configuring-fluent-bit/yaml/includes-section). -- `service`: Configures global properties of the Fluent Bit [service](../administration/configuring-fluent-bit/yaml/service-section). -- `pipeline`: Configures active [`inputs`, `filters`, and `outputs`](../administration/configuring-fluent-bit/yaml/pipeline-section). -- `parsers`: Defines [custom parsers](../administration/configuring-fluent-bit/yaml/parsers-section). -- `multiline_parsers`: Defines [custom multiline parsers](../administration/configuring-fluent-bit/yaml/multiline-parsers-section). -- `plugins`: Defines paths for [custom plugins](../administration/configuring-fluent-bit/yaml/plugins-section). -- `upstream_servers`: Defines [nodes](../administration/configuring-fluent-bit/yaml/upstream-servers-section) for output plugins. +- `env`: Configures [environment variables](./yaml/environment-variables-section). +- `includes`: Specifies additional YAML configuration files to [include as part of a parent file](./yaml/includes-section). +- `service`: Configures global properties of the Fluent Bit [service](./yaml/service-section). +- `pipeline`: Configures active [`inputs`, `filters`, and `outputs`](./yaml/pipeline-section). +- `parsers`: Defines [custom parsers](./yaml/parsers-section). +- `multiline_parsers`: Defines [custom multiline parsers](./yaml/multiline-parsers-section). +- `plugins`: Defines paths for [custom plugins](./yaml/plugins-section). +- `upstream_servers`: Defines [nodes](./yaml/upstream-servers-section) for output plugins. {% hint style="info" %} YAML configuration is used in the smoke tests for containers. An always-correct up-to-date example is here: . diff --git a/administration/configuring-fluent-bit/yaml/service-section.md b/administration/configuring-fluent-bit/yaml/service-section.md index 83949f53c..851bef2f7 100644 --- a/administration/configuring-fluent-bit/yaml/service-section.md +++ b/administration/configuring-fluent-bit/yaml/service-section.md @@ -11,22 +11,22 @@ The `service` section defines global properties of the service. The available co | `dns.mode` | Sets the primary transport layer protocol used by the asynchronous DNS resolver. Can be overridden on a per-plugin basis. | `UDP` | | `log_file` | Absolute path for an optional log file. By default, all logs are redirected to the standard error interface (`stderr`). | _none_ | | `log_level` | Sets the logging verbosity level. Possible values: `off`, `error`, `warn`, `info`, `debug`, and `trace`. Values are cumulative. For example, if `debug` is set, it will include `error`, `warning`, `info`, and `debug`. The `trace` mode is only available if Fluent Bit was built with the `WITH_TRACE` option enabled. | `info` | -| `parsers_file` | Path for a parsers configuration file. Multiple `parsers_file` entries can be defined within the section. Parsers can be declared directly in the [`parsers` section](../administration/configuring-fluent-bit/yaml/parsers-section.md) of YAML configuration files. | _none_ | -| `plugins_file` | Path for a `plugins` configuration file. This file specifies the paths to custom plugins (.so files) that Fluent Bit can load at runtime. Plugins can be declared directly in the [`plugins` section](../administration/configuring-fluent-bit/yaml/plugins-section.md) of YAML configuration files. | _none_ | -| `streams_file` | Path for the [stream processor](../stream-processing/overview.md) configuration file. This file defines the rules and operations for stream processing in Fluent Bit. Stream processor configurations can also be defined directly in the `streams` section of YAML configuration files. | _none_ | +| `parsers_file` | Path for a parsers configuration file. Multiple `parsers_file` entries can be defined within the section. Parsers can be declared directly in the [`parsers` section](./parsers-section.md) of YAML configuration files. | _none_ | +| `plugins_file` | Path for a `plugins` configuration file. This file specifies the paths to custom plugins (.so files) that Fluent Bit can load at runtime. Plugins can be declared directly in the [`plugins` section](./plugins-section.md) of YAML configuration files. | _none_ | +| `streams_file` | Path for the [stream processor](../../../stream-processing/overview.md) configuration file. This file defines the rules and operations for stream processing in Fluent Bit. Stream processor configurations can also be defined directly in the `streams` section of YAML configuration files. | _none_ | | `http_server` | Enables the built-in HTTP server. | `off` | | `http_listen` | Sets the listening interface for the HTTP Server when it's enabled. | `0.0.0.0` | | `http_port` | Sets the TCP port for the HTTP server. | `2020` | -| `hot_reload` | Enables [hot reloading](../administration/hot-reload.md) of configuration with SIGHUP. | `on` | +| `hot_reload` | Enables [hot reloading](../../../administration/hot-reload.md) of configuration with SIGHUP. | `on` | | `coro_stack_size` | Sets the coroutines stack size in bytes. The value must be greater than the page size of the running system. Setting the value too small (for example, `4096`) can cause coroutine threads to overrun the stack buffer. For best results, don't change this parameter from its default value. | `24576` | | `scheduler.cap` | Sets a maximum retry time in seconds. | `2000` | | `scheduler.base` | Sets the base of exponential backoff. | `5` | -| `json.convert_nan_to_null` | If enabled, `NaN` is converted to `null` when Fluent Bit converts msgpack to JSON. | `false` | +| `json.convert_nan_to_null` | If enabled, `NaN` is converted to `null` when Fluent Bit converts `msgpack` to JSON. | `false` | | `json.escape_unicode` | Controls how Fluent Bit serializes non‑ASCII / multi‑byte Unicode characters in JSON strings. When enabled, Unicode characters are escaped as `\uXXXX` sequences (characters outside BMP become surrogate pairs). When disabled, Fluent Bit emits raw UTF‑8 bytes. | `true` | | `sp.convert_from_str_to_num` | If enabled, the stream processor converts strings that represent numbers to a numeric type. | `true` | | `windows.maxstdio` | If specified, adjusts the limit of `stdio`. Only provided for Windows. Values from `512` to `2048` are allowed. | `512` | -### Storage configuration +## Storage configuration The following storage-related keys can be set in the `service` section: diff --git a/concepts/data-pipeline.md b/concepts/data-pipeline.md index 40c55295c..80ad9d54b 100644 --- a/concepts/data-pipeline.md +++ b/concepts/data-pipeline.md @@ -29,7 +29,7 @@ graph LR ## Buffer -The [buffering](./buffering.md) phase in the pipeline aims to provide a unified and persistent mechanism to store your data, using the primary in-memory model or the file system-based mode. +The [buffering](../pipeline/buffering.md) phase in the pipeline aims to provide a unified and persistent mechanism to store your data, using the primary in-memory model or the file system-based mode. ## Routing diff --git a/development/external-libraries.md b/development/external-libraries.md index 505d6eb19..b57277aeb 100644 --- a/development/external-libraries.md +++ b/development/external-libraries.md @@ -4,9 +4,9 @@ description: External libraries embedded in Fluent Bit # External libraries -Fluent Bit embeds several external libraries in its [/lib](https://github.com/fluent/fluent-bit/tree/master/lib) directory. These libraries are included to minimize external dependencies and maximize cross-platform compatibility. +Fluent Bit embeds several external libraries in its [`/lib`](https://github.com/fluent/fluent-bit/tree/master/lib) directory. These libraries are included to minimize external dependencies and maximize cross-platform compatibility. -## Core fluent libraries +## Core Fluent Bit libraries These libraries are developed and maintained by the Fluent Bit team: @@ -18,7 +18,7 @@ These libraries are developed and maintained by the Fluent Bit team: | [`CTraces`](https://github.com/fluent/ctraces) | Creates and manages trace contexts with encoding/decoding support for OpenTelemetry and other formats. | Apache 2.0 | | [`CProfiles`](https://github.com/fluent/cprofiles) | Creates and manages profiling data based on the OpenTelemetry Profiles schema. | Apache 2.0 | | [`Fluent OTel Proto`](https://github.com/fluent/fluent-otel-proto) | Provides C interfaces for OpenTelemetry protocol buffer definitions (common, resource, trace, logs, metrics). | Apache 2.0 | -| [`flb_libco`](https://github.com/fluent/flb_libco) | Fork of [libco](https://byuu.org/library/libco/) for cooperative multithreading (coroutines). Includes ARMv8 fixes and macOS support. | Internet Systems Consortium | +| [`flb_libco`](https://github.com/fluent/fluent-bit/tree/master/lib/flb_libco) | Fork of [`libco`](https://github.com/higan-emu/libco) for cooperative multithreading (coroutines). Includes ARMv8 fixes and macOS support. | Internet Systems Consortium | ## HTTP and networking @@ -42,7 +42,7 @@ These libraries are developed and maintained by the Fluent Bit team: | Library | Purpose | License | |---------|---------|---------| | [`Onigmo`](https://github.com/k-takata/Onigmo) | Regular expression library (fork of Oniguruma). Supports Perl 5.10+ expressions like `\K`, `\R`, and conditional patterns. Default regular expression engine for Ruby 2.0+. | BSD | -| [`tutf8e`](https://github.com/papplampe/tutf8e) | Tiny UTF-8 encoder supporting ISO-8859-x and Windows-125x character set conversions. | Massachusetts Institute of Technology (MIT) | +| [`tutf8e`](https://github.com/nigels-com/tutf8e) | Tiny UTF-8 encoder supporting ISO-8859-x and Windows-125x character set conversions. | Massachusetts Institute of Technology (MIT) | | [`simdutf`](https://github.com/simdutf/simdutf) | SIMD-accelerated Unicode validation and transcoding (UTF-8, UTF-16, UTF-32). | Apache 2.0 | ## Compression @@ -53,7 +53,7 @@ These libraries are developed and maintained by the Fluent Bit team: | [`Snappy`](https://github.com/google/snappy) | Fast compression/decompression library. Prioritizes speed over compression ratio. | BSD | | [`zstd`](https://github.com/facebook/zstd) | Zstandard compression providing high compression ratios at fast speeds. | BSD | -## Scripting and webassembly +## Scripting and WebAssembly | Library | Purpose | License | |---------|---------|---------| @@ -100,4 +100,3 @@ Some libraries are optional and can be enabled or disabled at build time: | `FLB_JEMALLOC` | `jemalloc` | Off (platform dependent) | For more information on build options, see [Build and install](../installation/downloads/source/build-and-install.md). - diff --git a/installation/downloads/docker.md b/installation/downloads/docker.md index 9c549a976..d9e593824 100644 --- a/installation/downloads/docker.md +++ b/installation/downloads/docker.md @@ -48,8 +48,6 @@ The following table describes the Linux container tags that are available on Doc | 4.1.1 | x86_64, arm64v8, arm32v7, s390x | Release [v4.1.1](https://fluentbit.io/announcements/v4.1.1/) | | 4.1.0-debug | x86_64, arm64v8, arm32v7, s390x | Debug images | | 4.1.0 | x86_64, arm64v8, arm32v7, s390x | Release [v4.1.0](https://fluentbit.io/announcements/v4.1.0/) | -| 4.0.13-debug | x86_64, arm64v8, arm32v7, s390x | Debug images | -| 4.0.13 | x86_64, arm64v8, arm32v7, s390x | Release [v4.0.13](https://fluentbit.io/announcements/v4.0.13/) | | 4.0.12-debug | x86_64, arm64v8, arm32v7, s390x | Debug images | | 4.0.12 | x86_64, arm64v8, arm32v7, s390x | Release [v4.0.12](https://fluentbit.io/announcements/v4.0.12/) | | 4.0.11-debug | x86_64, arm64v8, arm32v7, s390x | Debug images | diff --git a/installation/downloads/linux.md b/installation/downloads/linux.md index cb21e6675..de3667599 100644 --- a/installation/downloads/linux.md +++ b/installation/downloads/linux.md @@ -33,7 +33,7 @@ The GPG Key fingerprint of the old key is: F209 D876 2A60 CD49 E680 633B 4FF8 368B 6EA0 722A ``` -Refer to the [supported platform documentation](../getting-started-with-fluent-bit.md#supported-platforms-and-packages) to see which platforms are supported in each release. +Refer to the [supported platform documentation](../downloads.md#supported-platforms-and-packages) to see which platforms are supported in each release. ## Migration to Fluent Bit diff --git a/installation/downloads/source/build-and-install.md b/installation/downloads/source/build-and-install.md index d7d2c7df7..2d645ab59 100644 --- a/installation/downloads/source/build-and-install.md +++ b/installation/downloads/source/build-and-install.md @@ -14,7 +14,7 @@ To build and install Fluent Bit from source, you must also install the following - `libyaml-dev` - `pkg-config` -Additionally, certain [input](../pipeline/inputs.md) or [output](../pipeline/outputs.md) plugins might depend on additional components. For example, some plugins require Kafka. +Additionally, certain [input](../../../pipeline/inputs.md) or [output](../../../pipeline/outputs.md) plugins might depend on additional components. For example, some plugins require Kafka. ## Prepare environment @@ -305,7 +305,7 @@ The following table describes the output plugins available: | [`FLB_OUT_ORACLE_LOG_ANALYTICS`](../../../pipeline/outputs/oci-logging-analytics.md) | Enable Oracle Cloud Infrastructure Logging output plugin | `On` | | [`FLB_OUT_PGSQL`](../../../pipeline/outputs/postgresql.md) | Enable PostgreSQL output plugin | `Off` | | [`FLB_OUT_PLOT`](../../../pipeline/outputs/plot.md) | Enable Plot output plugin | `On` | -| [`FLB_OUT_PROMETHEUS_EXPORTER`](../../../pipeline/inputs/prometheus-exporter.md) | Enable Prometheus exporter output plugin | `On` | +| [`FLB_OUT_PROMETHEUS_EXPORTER`](../../../pipeline/outputs/prometheus-exporter.md) | Enable Prometheus exporter output plugin | `On` | | [`FLB_OUT_PROMETHEUS_REMOTE_WRITE`](../../../pipeline/inputs/prometheus-remote-write.md) | Enable Prometheus remote write output plugin | `On` | | [`FLB_OUT_S3`](../../../pipeline/outputs/s3.md) | Enable Amazon S3 output plugin | `On` | | [`FLB_OUT_SKYWALKING`](../../../pipeline/outputs/skywalking.md) | Enable Apache Skywalking output plugin | `On` | diff --git a/pipeline/inputs/tail.md b/pipeline/inputs/tail.md index 9396c81df..64eddb74c 100644 --- a/pipeline/inputs/tail.md +++ b/pipeline/inputs/tail.md @@ -10,7 +10,7 @@ The plugin supports the following configuration parameters: | Key | Description | Default | |:----|:------------|:--------| -| `buffer_chunk_size` | Set the initial buffer size to read file data. This value is used to increase buffer size. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification. | `32k` | +| `buffer_chunk_size` | Set the initial buffer size to read file data. This value is used to increase buffer size. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit.md#unit-sizes) specification. | `32k` | | `buffer_max_size` | Set the limit of the buffer size per monitored file. When a buffer needs to be increased, this value is used to restrict the memory buffer growth. If reading a file exceeds this limit, the file is removed from the monitored file list. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit.md#unit-sizes) specification. | `32k` | | `db` | Specify the database file to keep track of monitored files and offsets. Recommended to be unique per plugin. | _none_ | | `db.compare_filename` | This option determines whether to review both `inode` and `filename` when retrieving stored file information from the database. `true` verifies both `inode` and `filename`, while `false` checks only the `inode`. To review the `inode` and `filename` in the database, refer [see `keep_state`](#tailing-files-keeping-state). | `false` | diff --git a/pipeline/processors/filters.md b/pipeline/processors/filters.md index 6193b29e4..b6142d77c 100644 --- a/pipeline/processors/filters.md +++ b/pipeline/processors/filters.md @@ -44,7 +44,7 @@ pipeline: ### Lua -In this example configuration, an input plugin uses the [Lua](../pipeline/filters/lua.md) filter as a processor to add a new key `hostname` with the value `monox`. Then, an output plugin adds a new key named `output` with the value `new data`. +In this example configuration, an input plugin uses the [Lua](../filters/lua.md) filter as a processor to add a new key `hostname` with the value `monox`. Then, an output plugin adds a new key named `output` with the value `new data`. {% tabs %} {% tab title="fluent-bit.yaml" %}