Skip to content

[native] Add metrics for memory pushback mechanism#23422

Merged
xiaoxmeng merged 1 commit intoprestodb:masterfrom
tanjialiang:pushback_metrics
Aug 13, 2024
Merged

[native] Add metrics for memory pushback mechanism#23422
xiaoxmeng merged 1 commit intoprestodb:masterfrom
tanjialiang:pushback_metrics

Conversation

@tanjialiang
Copy link
Contributor

@tanjialiang tanjialiang commented Aug 12, 2024

Description

Adds 2 memory push back related metrics to the system. One pushback count, one pushback latency. This will help us understand how pushback mechanism works in our system and what could be improved.

== RELEASE NOTE ==

Following 2 metrics were added in Presto Native:

  • presto_cpp.memory_pushback_count -- Number of times memory pushback mechanism is triggered.
  • presto_cpp.memory_pushback_latency_ms -- Latency distribution of each memory pushback run in range of [0, 100s] and reports P50, P90, P99, and P100.

@tanjialiang tanjialiang marked this pull request as ready for review August 12, 2024 01:55
@tanjialiang tanjialiang requested a review from a team as a code owner August 12, 2024 01:55
Copy link
Contributor

@xiaoxmeng xiaoxmeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tanjialiang thanks!

if (config_.systemMemPushbackEnabled &&
systemUsedMemoryBytes() > config_.systemMemLimitBytes) {
pushbackMemory();
RECORD_METRIC_VALUE(kCounterMemoryPushbackCount);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about handle this inside pushbackMemory()? Thanks!

@aditi-pandit
Copy link
Contributor

@tanjialiang : Please add a release note for this change since you are adding user-visible metrics.

xiaoxmeng
xiaoxmeng previously approved these changes Aug 12, 2024
Copy link
Contributor

@xiaoxmeng xiaoxmeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tanjialiang LGTM. Thanks for the update!


/// ================== Memory Pushback Counters =================

/// Number of times memory pushback mechanism is triggered.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider to add document for Presitssimo metrics as Velox does in followup. Thanks!

@xiaoxmeng xiaoxmeng merged commit e782f02 into prestodb:master Aug 13, 2024
@tdcmeehan tdcmeehan mentioned this pull request Aug 23, 2024
34 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants