Skip to content

MINOR: add missing docs for record-e2e-latency metrics #10251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ableegoldman merged 7 commits into from
Mar 4, 2021
Merged

MINOR: add missing docs for record-e2e-latency metrics #10251

Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a4c9250
add missing metrics
ableegoldman Mar 3, 2021
a95d25e
add missing active-process-ratio
ableegoldman Mar 3, 2021
eb522d7
standardize formatting
ableegoldman Mar 3, 2021
0dccb07
fix typo in release script on the side
ableegoldman Mar 3, 2021
d306f53
fix typo, break up long lines
ableegoldman Mar 4, 2021
bf7f8d7
add TRACE level
ableegoldman Mar 4, 2021
e7e3229
fix name
ableegoldman Mar 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 38 additions & 3 deletions docs/ops.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2277,8 +2277,8 @@ <h5 class="anchor-heading"><a id="kafka_streams_thread_monitoring" class="anchor
</table>

<h5 class="anchor-heading"><a id="kafka_streams_task_monitoring" class="anchor-link"></a><a href="#kafka_streams_task_monitoring">Task Metrics</a></h5>
All of the following metrics have a recording level of <code>debug</code>, except for metrics
dropped-records-rate and dropped-records-total which have a recording level of <code>info</code>:
All of the following metrics have a recording level of <code>debug</code>, except for the
dropped-records-rated-*, active-process-ratio, and record-e2e-latency-* metrics which have a recording level of <code>info</code>:
<table class="data-table">
<tbody>
<tr>
Expand Down Expand Up @@ -2356,6 +2356,26 @@ <h5 class="anchor-heading"><a id="kafka_streams_task_monitoring" class="anchor-l
<td>The total number of records dropped within this task.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>active-process-ratio</td>

@ableegoldman ableegoldman Mar 3, 2021

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noticed that we seem to be missing this task-level metric as well. Might be good to take stock of all the metrics at some point and make sure they're documented clearly, especially since it's easy to miss this doc which is outside of the usual Streams docs

<td>The total number of records dropped within this task.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>record-e2e-latency-avg</td>
<td>The average end-to-end latency of a record, measured by comparing the record timestamp with the system time when it has been fully processed by the node.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>record-e2e-latency-max</td>
<td>The maximum end-to-end latency of a record, measured by comparing the record timestamp with the system time when it has been fully processed by the node.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
<tr>
<td>record-e2e-latency-min</td>
<td>The minimum end-to-end latency of a record, measured by comparing the record timestamp with the system time when it has been fully processed by the node.</td>
<td>kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)</td>
</tr>
</tbody>
</table>

Expand Down Expand Up @@ -2394,7 +2414,7 @@ <h5 class="anchor-heading"><a id="kafka_streams_node_monitoring" class="anchor-l
</table>

<h5 class="anchor-heading"><a id="kafka_streams_store_monitoring" class="anchor-link"></a><a href="#kafka_streams_store_monitoring">State Store Metrics</a></h5>
All of the following metrics have a recording level of <code>debug</code>. Note that the <code>store-scope</code> value is specified in <code>StoreSupplier#metricsScope()</code> for user's customized
All of the following metrics have a recording level of <code>debug</code>, except for the <code>record-e2e-latency-*</code> metrics which have a recording level <code>trace></code>. Note that the <code>store-scope</code> value is specified in <code>StoreSupplier#metricsScope()</code> for user's customized
state stores; for built-in state stores, currently we have:
<ul>
<li><code>in-memory-state</code></li>
Expand Down Expand Up @@ -2570,6 +2590,21 @@ <h5 class="anchor-heading"><a id="kafka_streams_store_monitoring" class="anchor-
<td>The maximum number of records buffered over the sampling window.</td>
<td>kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),in-memory-suppression-id=([-.\w]+)</td>
</tr>
<tr>
<td>record-e2e-latency-avg</td>
<td>The average end-to-end latency of a record, measured by comparing the record timestamp with the system time when it has been fully processed by the node.</td>
<td>kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
</tr>
<tr>
<td>record-e2e-latency-max</td>
<td>The maximum end-to-end latency of a record, measured by comparing the record timestamp with the system time when it has been fully processed by the node.</td>
<td>kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
</tr>
<tr>
<td>record-e2e-latency-min</td>
<td>The minimum end-to-end latency of a record, measured by comparing the record timestamp with the system time when it has been fully processed by the node.</td>
<td>kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td>
</tr>
</tbody>
</table>

Expand Down