Skip to content

[DOCS-13090] Add OP utilization metrics#33838

Merged
maycmlee merged 8 commits intomasterfrom
may/add-op-comp-buffer-metrics
Jan 22, 2026
Merged

[DOCS-13090] Add OP utilization metrics#33838
maycmlee merged 8 commits intomasterfrom
may/add-op-comp-buffer-metrics

Conversation

@maycmlee
Copy link
Contributor

@maycmlee maycmlee commented Jan 13, 2026

What does this PR do? What is the motivation?

Merge instructions

Merge readiness:

  • Ready for merge

For Datadog employees:

Your branch name MUST follow the <name>/<description> convention and include the forward slash (/). Without this format, your pull request will not pass CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.

If your branch doesn't follow this format, rename it or create a new branch and PR.

[6/5/2025] Merge queue has been disabled on the documentation repo. If you have write access to the repo, the PR has been reviewed by a Documentation team member, and all of the required checks have passed, you can use the Squash and Merge button to merge the PR. If you don't have write access, or you need help, reach out in the #documentation channel in Slack.

Additional notes

@maycmlee maycmlee requested a review from a team as a code owner January 13, 2026 15:25
@maycmlee maycmlee added the WORK IN PROGRESS No review needed, it's a wip ;) label Jan 13, 2026
@github-actions github-actions bot added the Architecture Everything related to the Doc backend label Jan 13, 2026
@github-actions
Copy link
Contributor

maycmlee and others added 2 commits January 13, 2026 10:52
…ing/pipeline_usage_metrics.md

Co-authored-by: Bruce Guenter <bruce@untroubled.org>
: Events discarded from the buffer (for example, due to overflow).

`pipelines.source_buffer_max_event_size`
: Maximum event size for a source's buffer.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an unfortunate choice of metric name, as it actually means the maximum buffer size in terms of number of events. It probably should have been max_size_events which would follow other standard metric naming conventions. I will ask the Vector team their thoughts about renaming this. It'll be a breaking change but we haven't really advertised this yet.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this followed the Vector buffer metric naming spec which contradicts the Vector metric naming spec

Copy link
Contributor Author

@maycmlee maycmlee Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh okay, that actually makes more sense. If you and Vector want to make the name change, let me know if you I should hold off publishing this.

Suggested change
: Maximum event size for a source's buffer.
: A source's maximum buffer size, defined as the number of events it can hold.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed these metrics since the names are getting changed and released with 2.13 release:

  • pipelines.source_buffer_max_event_size
  • pipelines.transform_buffer_max_event_size

Copy link
Contributor Author

@maycmlee maycmlee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bruceg! I followed up and made edits, could you confirm they look good or if they need further changes?

: Events discarded from the buffer (for example, due to overflow).

`pipelines.source_buffer_max_event_size`
: Maximum event size for a source's buffer.
Copy link
Contributor Author

@maycmlee maycmlee Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh okay, that actually makes more sense. If you and Vector want to make the name change, let me know if you I should hold off publishing this.

Suggested change
: Maximum event size for a source's buffer.
: A source's maximum buffer size, defined as the number of events it can hold.

@maycmlee maycmlee removed the WORK IN PROGRESS No review needed, it's a wip ;) label Jan 22, 2026
: **Metric**: `pipelines.component_timed_out_requests_total`
: **Description**: The number of requests that timed out for sources that send events to the Worker in batches using HTTP requests.
: **Available for**: HTTP-based sources that have a configured timeout, such as the Datadog Agent.

Copy link
Contributor

@iadjivon iadjivon Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be addressed in a different PR, but visually, I wonder if it might be best to emulate the Buffer metrics (when buffering is enabled) format without mentioning
Metric, Description, and Available for

Copy link
Contributor Author

@maycmlee maycmlee Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @iadjivon! I don't quite understand. Are you saying to put the metric name as the header instead of Timed out request?

@maycmlee maycmlee merged commit 350f8e3 into master Jan 22, 2026
19 checks passed
@maycmlee maycmlee deleted the may/add-op-comp-buffer-metrics branch January 22, 2026 22:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Architecture Everything related to the Doc backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants