Skip to content

Connector circuit breaker metrics#40729

Merged
espadolini merged 4 commits intomasterfrom
espadolini/joins-and-breakers
Apr 22, 2024
Merged

Connector circuit breaker metrics#40729
espadolini merged 4 commits intomasterfrom
espadolini/joins-and-breakers

Conversation

@espadolini
Copy link
Copy Markdown
Contributor

This PR adds a teleport_breaker_connector_executions_total prom metric to count circuit breaker executions for the circuit breakers used by Teleport connectors, labeled by system role, circuit breaker state (standby, tripped and recovering) and execution success (true or false), and instruments the connectors to feed the metric.

In addition, this adds additional logging for failed instance/bot joins to the cluster, which currently log a message before actually doing anything, and then don't log anything on error - which made a recent sev1 harder to diagnose than it should've been.

changelog: added a new Prometheus metric to track requests initiated by Teleport against the control plane API

@espadolini espadolini changed the title Espadolini/joins and breakers Connector circuit breaker metrics Apr 19, 2024
@github-actions github-actions Bot requested review from GavinFrazar and avatus April 19, 2024 19:21
Comment thread api/breaker/breaker.go
Comment thread lib/service/connect.go
@espadolini espadolini requested a review from rosstimothy April 19, 2024 20:14
@public-teleport-github-review-bot public-teleport-github-review-bot Bot removed the request for review from GavinFrazar April 20, 2024 18:07
@espadolini espadolini added this pull request to the merge queue Apr 22, 2024
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 22, 2024
@espadolini espadolini added this pull request to the merge queue Apr 22, 2024
Merged via the queue into master with commit 52bc80b Apr 22, 2024
@espadolini espadolini deleted the espadolini/joins-and-breakers branch April 22, 2024 09:35
@public-teleport-github-review-bot
Copy link
Copy Markdown

@espadolini See the table below for backport results.

Branch Result
branch/v13 Failed
branch/v14 Failed
branch/v15 Create PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants