diff --git a/processor/memorylimiterprocessor/README.md b/processor/memorylimiterprocessor/README.md index 492f0e1d73b..f12f19b1c6c 100644 --- a/processor/memorylimiterprocessor/README.md +++ b/processor/memorylimiterprocessor/README.md @@ -38,6 +38,13 @@ The processor will enter memory limited mode and will start refusing the data wh memory usage exceeds the soft limit. This is done by returning errors to the preceding component in the pipeline that made the ConsumeLogs/Trace/Metrics function call. +> Warning: Incoming data can consume additional memory in a Collector before the +> memory limiter processor is able to reject it. Be sure to consider this when +> setting your limits, particularly for non-OTLP receivers. +> +> See for +> more. + In memory limited mode the error returned by ConsumeLogs/Trace/Metrics function is a non-permanent error. When receivers see this error they are expected to retry sending the same data. The receivers may also apply backpressure to their own data sources @@ -79,12 +86,33 @@ memory check interval. Otherwise, memory usage may exceed the hard limit, even i A good starting point for `spike_limit_mib` is 20% of the hard limit. Bigger `spike_limit_mib` values may be necessary for spiky traffic or for longer check intervals. +It's recommended to coordinate the hard memory limit with the host environment +of the Collector. As always, you know your environment best, so use your own +judgement to determine the best configuration for your situation. However, the +following points are worth considering when configuring the processor: + +For containerized environments or other similar environments that support +setting memory restrictions on the Collector, `limit_percentage` should +generally be used, which allows the host environment to determine the +Collector's max memory allocation without tying the Collector's own config to +its deployment config. The percentage should be large enough to use the majority +of the Collector's allocated memory, but not so much that it risks running out +of memory. + +For bare metal or virtualized environments where the Collector's memory is not +constrained by the host environment, it is recommended to use `limit_mib` for +environments where you know the rough data throughput and memory consumption you +expect for a given Collector, regardless of the memory capacity of the machine +it runs on. If the Collector's memory consumption is expected to scale with its +host machine's memory capacity, `limit_percentage` may be more appropriate. ## Configuration -Please refer to [memorylimiter.go](../../internal/memorylimiter/memorylimiter.go) for the config spec. +Please refer to [config.go](../../internal/memorylimiter/config.go) for the config spec. + +The following configuration options are available. Note that one of `limit_mib` +or `limit_percentage` must be set. -The following configuration options **must be changed**: - `check_interval` (default = 0s): Time between measurements of memory usage. The recommended value is 1 second. If the expected traffic to the Collector is very spiky then decrease the `check_interval` @@ -94,7 +122,7 @@ allocated by the process heap. Note that typically the total memory usage of process will be about 50MiB higher than this value. This defines the hard limit. - `spike_limit_mib` (default = 20% of `limit_mib`): Maximum spike expected between the measurements of memory usage. The value must be less than `limit_mib`. The soft limit -value will be equal to (limit_mib - spike_limit_mib). +value will be equal to (`limit_mib - spike_limit_mib`). The recommended value for `spike_limit_mib` is about 20% `limit_mib`. - `limit_percentage` (default = 0): Maximum amount of total memory targeted to be allocated by the process heap. This configuration is supported on Linux systems with cgroups @@ -134,6 +162,3 @@ On a machine with 1000 MiB total memory available: - Hard limit will be set to 1000 * 0.80 = **800 MiB**. - Soft limit will be set to 1000 * 0.80 - 1000 * 0.15 = 1000 * 0.65 = **650 MiB**. - -Refer to [config.yaml](../../internal/memorylimiter/testdata/config.yaml) for detailed -examples on using the processor.