Limit tshd notifications to one per kind per root cluster#62031
Merged
Limit tshd notifications to one per kind per root cluster#62031
Conversation
This necessitated refactoring the state of the service from an array to a map.
ravicious
force-pushed
the
r7s/notification-key
branch
from
92b93ef to
40240c0
Compare
gzdunek
approved these changes
Dec 8, 2025
Contributor
gzdunek
left a comment
gzdunek
left a comment
There was a problem hiding this comment.
Nice work, this mechanism is so elegant.
Member
Author
|
@kimlisa ping |
avatus
approved these changes
Dec 22, 2025
Contributor
|
@ravicious See the table below for backport results.
|
This was referenced Jan 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #51336.
changelog: Fixed an issue where Teleport Connect could generate a flurry of notifications about not being able to connect to a resource
You can see in Bartosz's report that Connect can generate a bunch of notifications. In this case it's about an app proxy not being able to connect to the app because the user did not relogin within 1 minute. This can happen when a program repeatedly tries to connect to a proxied resource over the course of a night. When you open Connect the next day, you're greeted by a flurry of notifications that's impossible to close manually because there's so many of them.
My solution to this is to add a
keyproperty to a notification. Notifications with the same key overwrite each other. This enables us to limit tshd notifications to one per kind per root cluster. This way if there are multiple clients hammering multiple local proxies all belonging to the same cluster, the user is only ever going to see a single notification in Connect.Sure, we lose some information, but the notifications are not supposed to be detailed diagnostic reports, merely a sign that something went wrong. If the user sees a single notification about not being able to relogin, they should still know what to do.
There are already callsites where we manually apply the one notification limit (e.g., in
AppContext). The newkeyproperty will remove the need to store the notification ID manually and instead the callsite will be able to just pass constant key to the notifications service.I tested this by logging in as a user with short TTL, then starting a Postgres gateway, waiting for the cert to expire and then attempting to connect to the proxy through multiple psql invocations in separate shell sessions.