Skip to content

Commit 9f46b6f

Browse files
Fix GossipMembersMismatch misfiring with remote ruler eval (#2159)
* Fix GossipMembersMismatch misfiring with remote ruler eval The alert doesn't take into account the queries that now have ruler- prepended to their name. This PR fixes it Signed-off-by: Dimitar Dimitrov <[email protected]> * Simpler regexes * Update computed mixin
1 parent d58da4a commit 9f46b6f

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

operations/mimir-mixin-compiled/alerts.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -352,7 +352,7 @@ groups:
352352
message: Mimir instance {{ $labels.pod }} in {{ $labels.cluster }}/{{ $labels.namespace
353353
}} sees incorrect number of gossip members.
354354
expr: |
355-
avg by (cluster, namespace) (memberlist_client_cluster_members_count) != sum by (cluster, namespace) (up{job=~".+/(alertmanager|compactor|distributor|ingester.*|querier.*|ruler|store-gateway.*|cortex|mimir)"})
355+
avg by (cluster, namespace) (memberlist_client_cluster_members_count) != sum by (cluster, namespace) (up{job=~".+/(alertmanager|compactor|distributor|ingester.*|querier.*|ruler|ruler-querier.*|store-gateway.*|cortex|mimir)"})
356356
for: 15m
357357
labels:
358358
severity: warning

operations/mimir-mixin/config.libsonnet

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@
3030
ruler_query_frontend: '(ruler-query-frontend.*)', // Match also custom ruler-query-frontend deployments.
3131
query_scheduler: 'query-scheduler.*', // Not part of single-binary. Match also custom query-scheduler deployments.
3232
ruler_query_scheduler: 'ruler-query-scheduler.*', // Not part of single-binary. Match also custom query-scheduler deployments.
33-
ring_members: ['alertmanager', 'compactor', 'distributor', 'ingester.*', 'querier.*', 'ruler', 'store-gateway.*', 'cortex', 'mimir'],
33+
ring_members: ['alertmanager', 'compactor', 'distributor', 'ingester.*', 'querier.*', 'ruler', 'ruler-querier.*', 'store-gateway.*', 'cortex', 'mimir'],
3434
store_gateway: '(store-gateway.*|cortex|mimir)', // Match also per-zone store-gateway deployments.
3535
gateway: '(gateway|cortex-gw|cortex-gw-internal)',
3636
compactor: 'compactor.*|cortex|mimir', // Match also custom compactor deployments.

0 commit comments

Comments
 (0)