Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add burn rate graphs to APIServer dashboard #735

Open
metalmatze opened this issue Jan 25, 2022 · 1 comment
Open

Add burn rate graphs to APIServer dashboard #735

metalmatze opened this issue Jan 25, 2022 · 1 comment
Labels
enhancement New feature or request stale

Comments

@metalmatze
Copy link
Member

Whenever a service is burning too much of its error budget the alerts fire.
Because we have multi error burn rates for the API server it would be helpful to visualize them in 4 graphs:

burnrates

For example, in this particular case, my second most critical alert was flaky and here you can clearly see why (second graph).
In my case, since I sadly cannot change my cloud providers api server (other than escalating), the only option is to reduce the objective to a more realistic target (it's pretty much just the latencies).
Not only should the warning alerts stop firing (since less error budget is burned) but also that second alert should stop being as flaky.

The queries for the first panel are:

sum(apiserver_request:burnrate5m)
sum(apiserver_request:burnrate1h)

In case someone wants to give it a shot implementing these 4 graphs with jsonnet tooling (my screenshot is just written by hand and doesn't adjust to individual objectives), I left this issue. Maybe I'll get to it soon, but until then I'm happy to help contributors instead

@metalmatze metalmatze added the enhancement New feature or request label Jan 25, 2022
Copy link

github-actions bot commented Oct 4, 2024

This issue has not had any activity in the past 30 days, so the
stale label has been added to it.

  • The stale label will be removed if there is new activity
  • The issue will be closed in 7 days if there is no new activity
  • Add the keepalive label to exempt this issue from the stale check action

Thank you for your contributions!

@github-actions github-actions bot added the stale label Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request stale
Projects
None yet
Development

No branches or pull requests

1 participant