Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge metrics across workers #1119

Merged
merged 2 commits into from
Jul 10, 2024
Merged

merge metrics across workers #1119

merged 2 commits into from
Jul 10, 2024

Conversation

Keksoj
Copy link
Member

@Keksoj Keksoj commented Jul 8, 2024

Up until now, querying metrics would yield all metrics of all workers, which means cluster metrics are consultable only this way:

  • worker 1
    • proxy metrics for worker 1
    • cluster metrics for cluster "my-app" in worker 1
  • worker 2
    • proxy metrics for worker 2
    • cluster metrics for cluster "my-app" in worker 2

The default behaviour is now to take all worker metrics from the AggregatedMetrics object, merge them by worker id or cluster id, and move them to the field AggregatedMetrics::clusters or AggregatedMetrics::proxying, which allows to display metrics this way:

  • proxying
    • proxy metrics of all workers, accumulated
  • clusters
    • cluster metrics for cluster "my-app", accumulated

The behaviour is avoidable with the option "metrics get --workers"

Note: the function merge_metrics does not merge time metrics (percentiles), they are discarded. Prometheus histograms should be mergeable though, if bucket size is consistent.

@Keksoj Keksoj added the metrics label Jul 8, 2024
@Keksoj Keksoj self-assigned this Jul 8, 2024
@Keksoj Keksoj force-pushed the flatten-cluster-metrics branch 3 times, most recently from 48f7645 to f6f7ed3 Compare July 8, 2024 09:37
@Keksoj Keksoj changed the title merge cluster metrics across workers in AggregatedMetrics merge cluster metrics across workers Jul 8, 2024
@Keksoj Keksoj changed the title merge cluster metrics across workers merge metrics across workers Jul 8, 2024
@Keksoj Keksoj changed the title merge metrics across workers merge across workers Jul 8, 2024
@Keksoj Keksoj changed the title merge across workers merge metrics across workers Jul 8, 2024
The default behaviour is now to take all metrics from the
AggregatedMetrics object, merge them by cluster id, and
move them to the field AggregatedMetrics::clusters.
The behaviour is avoidable with the option "metrics get --workers"

Note: the function merge_cluster_metrics
does not merge time metrics (percentiles), they are discarded.
Prometheus histograms should be mergeable though, if bucket size is
consistent.
@Keksoj Keksoj requested a review from llenotre July 9, 2024 16:06
@Wonshtrum Wonshtrum merged commit f14a14d into main Jul 10, 2024
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants