[DOCS-13090] Add OP utilization metrics#33838
Conversation
content/en/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md
Show resolved
Hide resolved
layouts/shortcodes/observability_pipelines/metrics/buffer.en.md
Outdated
Show resolved
Hide resolved
layouts/shortcodes/observability_pipelines/metrics/buffer.en.md
Outdated
Show resolved
Hide resolved
Preview links (active after the
|
…ing/pipeline_usage_metrics.md Co-authored-by: Bruce Guenter <bruce@untroubled.org>
| : Events discarded from the buffer (for example, due to overflow). | ||
|
|
||
| `pipelines.source_buffer_max_event_size` | ||
| : Maximum event size for a source's buffer. |
There was a problem hiding this comment.
This is an unfortunate choice of metric name, as it actually means the maximum buffer size in terms of number of events. It probably should have been max_size_events which would follow other standard metric naming conventions. I will ask the Vector team their thoughts about renaming this. It'll be a breaking change but we haven't really advertised this yet.
There was a problem hiding this comment.
Note that this followed the Vector buffer metric naming spec which contradicts the Vector metric naming spec
There was a problem hiding this comment.
Ahh okay, that actually makes more sense. If you and Vector want to make the name change, let me know if you I should hold off publishing this.
| : Maximum event size for a source's buffer. | |
| : A source's maximum buffer size, defined as the number of events it can hold. |
There was a problem hiding this comment.
There was a problem hiding this comment.
Removed these metrics since the names are getting changed and released with 2.13 release:
pipelines.source_buffer_max_event_sizepipelines.transform_buffer_max_event_size
content/en/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md
Show resolved
Hide resolved
content/en/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md
Outdated
Show resolved
Hide resolved
| : Events discarded from the buffer (for example, due to overflow). | ||
|
|
||
| `pipelines.source_buffer_max_event_size` | ||
| : Maximum event size for a source's buffer. |
There was a problem hiding this comment.
Ahh okay, that actually makes more sense. If you and Vector want to make the name change, let me know if you I should hold off publishing this.
| : Maximum event size for a source's buffer. | |
| : A source's maximum buffer size, defined as the number of events it can hold. |
layouts/shortcodes/observability_pipelines/metrics/buffer.en.md
Outdated
Show resolved
Hide resolved
layouts/shortcodes/observability_pipelines/metrics/buffer.en.md
Outdated
Show resolved
Hide resolved
layouts/shortcodes/observability_pipelines/metrics/buffer.en.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md
Outdated
Show resolved
Hide resolved
| : **Metric**: `pipelines.component_timed_out_requests_total` | ||
| : **Description**: The number of requests that timed out for sources that send events to the Worker in batches using HTTP requests. | ||
| : **Available for**: HTTP-based sources that have a configured timeout, such as the Datadog Agent. | ||
|
|
There was a problem hiding this comment.
This can be addressed in a different PR, but visually, I wonder if it might be best to emulate the Buffer metrics (when buffering is enabled) format without mentioning
Metric, Description, and Available for
There was a problem hiding this comment.
Hi @iadjivon! I don't quite understand. Are you saying to put the metric name as the header instead of Timed out request?
content/en/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md
Outdated
Show resolved
Hide resolved
…ing/pipeline_usage_metrics.md
What does this PR do? What is the motivation?
Merge instructions
Merge readiness:
For Datadog employees:
Your branch name MUST follow the
<name>/<description>convention and include the forward slash (/). Without this format, your pull request will not pass CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.If your branch doesn't follow this format, rename it or create a new branch and PR.
[6/5/2025] Merge queue has been disabled on the documentation repo. If you have write access to the repo, the PR has been reviewed by a Documentation team member, and all of the required checks have passed, you can use the Squash and Merge button to merge the PR. If you don't have write access, or you need help, reach out in the #documentation channel in Slack.
Additional notes