Refactor healthcheck for Kubernetes extensibility with HealthChecker interface#59396
Merged
Refactor healthcheck for Kubernetes extensibility with HealthChecker interface#59396
healthcheck for Kubernetes extensibility with HealthChecker interface#59396Conversation
rana
force-pushed
the
rana/kube-healthchecks-5
branch
from
b9182f6 to
d303c2f
Compare
rana
added
kubernetes-access
no-changelog
rana
requested review from
GavinFrazar,
Joerger,
Tener,
rosstimothy and
tigrato
github-actions
bot
added
database-access
Tener
reviewed
Sep 22, 2025
Joerger
reviewed
Sep 22, 2025
rana
force-pushed
the
rana/kube-healthchecks-5
branch
from
d303c2f to
4c32d21
Compare
GavinFrazar
approved these changes
Sep 23, 2025
Contributor
GavinFrazar
left a comment
GavinFrazar
left a comment
There was a problem hiding this comment.
just a couple nitpicks but LGTM overall
rana
force-pushed
the
rana/kube-healthchecks-5
branch
2 times, most recently
from
3361a42 to
1893c06
Compare
Tener
approved these changes
Sep 24, 2025
public-teleport-github-review-bot
bot
removed request for
rosstimothy and
tigrato
rana
force-pushed
the
rana/kube-healthchecks-5
branch
2 times, most recently
from
906bfda to
cb973fe
Compare
rana
changed the title
TargetDialerhealthcheck for Kubernetes extensibility with HealthChecker interface
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
rana
force-pushed
the
rana/kube-healthchecks-5
branch
from
cb973fe to
d30035f
Compare
…er` interface
The main intent of refactoring is to provide health check extensibility for Kubernetes while supporting the existing DB health checks. A `HealthChecker` interface is added to support the different health check approaches of DBs and Kubernetes. Existing DB TCP health check logic is moved to a new `TargetDialer` struct.
Changes:
- Added `HealthChecker` interface with two functions:
- `CheckHealth(ctx context.Context) ([]string, error)`
- `GetProtocol() types.TargetHealthProtocol`
- Added `TargetDialer` struct which encapsulates existing TCP health check logic
- Changed `Target` struct to use the `HealthChecker` interface
- Changed `worker.checkHealth` to call the new `CheckHealth` function
- Removed a `protocol` field from `healthCheckConfig`
- Added `TargetHealthProtocolHTTP` for use with Kubernetes health checks
- Moved and renamed test `Test_dialEndpoints` to `TestTargetDialer_dialEndpoints`
- Added files `net.go` and `net_test.go` for `TargetDialer`
Part of #58413
rana
force-pushed
the
rana/kube-healthchecks-5
branch
from
d30035f to
7c2c225
Compare
rana
added a commit
that referenced
this pull request
Oct 23, 2025
…er` interface (#59396) The main intent of refactoring is to provide health check extensibility for Kubernetes while supporting the existing DB health checks. A `HealthChecker` interface is added to support the different health check approaches of DBs and Kubernetes. Existing DB TCP health check logic is moved to a new `TargetDialer` struct. Changes: - Added `HealthChecker` interface with two functions: - `CheckHealth(ctx context.Context) ([]string, error)` - `GetProtocol() types.TargetHealthProtocol` - Added `TargetDialer` struct which encapsulates existing TCP health check logic - Changed `Target` struct to use the `HealthChecker` interface - Changed `worker.checkHealth` to call the new `CheckHealth` function - Removed a `protocol` field from `healthCheckConfig` - Added `TargetHealthProtocolHTTP` for use with Kubernetes health checks - Moved and renamed test `Test_dialEndpoints` to `TestTargetDialer_dialEndpoints` - Added files `net.go` and `net_test.go` for `TargetDialer` Part of #58413
37 tasks
rana
added a commit
that referenced
this pull request
Oct 27, 2025
…er` interface (#59396) The main intent of refactoring is to provide health check extensibility for Kubernetes while supporting the existing DB health checks. A `HealthChecker` interface is added to support the different health check approaches of DBs and Kubernetes. Existing DB TCP health check logic is moved to a new `TargetDialer` struct. Changes: - Added `HealthChecker` interface with two functions: - `CheckHealth(ctx context.Context) ([]string, error)` - `GetProtocol() types.TargetHealthProtocol` - Added `TargetDialer` struct which encapsulates existing TCP health check logic - Changed `Target` struct to use the `HealthChecker` interface - Changed `worker.checkHealth` to call the new `CheckHealth` function - Removed a `protocol` field from `healthCheckConfig` - Added `TargetHealthProtocolHTTP` for use with Kubernetes health checks - Moved and renamed test `Test_dialEndpoints` to `TestTargetDialer_dialEndpoints` - Added files `net.go` and `net_test.go` for `TargetDialer` Part of #58413
rana
added a commit
that referenced
this pull request
Oct 29, 2025
…er` interface (#59396) The main intent of refactoring is to provide health check extensibility for Kubernetes while supporting the existing DB health checks. A `HealthChecker` interface is added to support the different health check approaches of DBs and Kubernetes. Existing DB TCP health check logic is moved to a new `TargetDialer` struct. Changes: - Added `HealthChecker` interface with two functions: - `CheckHealth(ctx context.Context) ([]string, error)` - `GetProtocol() types.TargetHealthProtocol` - Added `TargetDialer` struct which encapsulates existing TCP health check logic - Changed `Target` struct to use the `HealthChecker` interface - Changed `worker.checkHealth` to call the new `CheckHealth` function - Removed a `protocol` field from `healthCheckConfig` - Added `TargetHealthProtocolHTTP` for use with Kubernetes health checks - Moved and renamed test `Test_dialEndpoints` to `TestTargetDialer_dialEndpoints` - Added files `net.go` and `net_test.go` for `TargetDialer` Part of #58413
rana
added a commit
that referenced
this pull request
Oct 29, 2025
…er` interface (#59396) The main intent of refactoring is to provide health check extensibility for Kubernetes while supporting the existing DB health checks. A `HealthChecker` interface is added to support the different health check approaches of DBs and Kubernetes. Existing DB TCP health check logic is moved to a new `TargetDialer` struct. Changes: - Added `HealthChecker` interface with two functions: - `CheckHealth(ctx context.Context) ([]string, error)` - `GetProtocol() types.TargetHealthProtocol` - Added `TargetDialer` struct which encapsulates existing TCP health check logic - Changed `Target` struct to use the `HealthChecker` interface - Changed `worker.checkHealth` to call the new `CheckHealth` function - Removed a `protocol` field from `healthCheckConfig` - Added `TargetHealthProtocolHTTP` for use with Kubernetes health checks - Moved and renamed test `Test_dialEndpoints` to `TestTargetDialer_dialEndpoints` - Added files `net.go` and `net_test.go` for `TargetDialer` Part of #58413
rana
added a commit
that referenced
this pull request
Oct 29, 2025
…er` interface (#59396) The main intent of refactoring is to provide health check extensibility for Kubernetes while supporting the existing DB health checks. A `HealthChecker` interface is added to support the different health check approaches of DBs and Kubernetes. Existing DB TCP health check logic is moved to a new `TargetDialer` struct. Changes: - Added `HealthChecker` interface with two functions: - `CheckHealth(ctx context.Context) ([]string, error)` - `GetProtocol() types.TargetHealthProtocol` - Added `TargetDialer` struct which encapsulates existing TCP health check logic - Changed `Target` struct to use the `HealthChecker` interface - Changed `worker.checkHealth` to call the new `CheckHealth` function - Removed a `protocol` field from `healthCheckConfig` - Added `TargetHealthProtocolHTTP` for use with Kubernetes health checks - Moved and renamed test `Test_dialEndpoints` to `TestTargetDialer_dialEndpoints` - Added files `net.go` and `net_test.go` for `TargetDialer` Part of #58413
github-merge-queue bot
pushed a commit
that referenced
this pull request
Oct 29, 2025
* Add protobufs for Kubernetes health checks (#58415) - Add Kubernetes label matchers to `Matcher` for `HealthCheckConfig` - Add message `KubernetesServerStatusV3` - Add `status` field to `KubernetesServerV3` - Add `target_health` field to `Kube` for UI - Regenerate Terraform schema and docs for `HealthCheckConfig` - Add Kubernetes label matchers to Terraform test `TestImportHealthCheckConfig` Relates to #58413 Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com> * Refactor `healthcheck` for Kubernetes extensibility with `HealthChecker` interface (#59396) The main intent of refactoring is to provide health check extensibility for Kubernetes while supporting the existing DB health checks. A `HealthChecker` interface is added to support the different health check approaches of DBs and Kubernetes. Existing DB TCP health check logic is moved to a new `TargetDialer` struct. Changes: - Added `HealthChecker` interface with two functions: - `CheckHealth(ctx context.Context) ([]string, error)` - `GetProtocol() types.TargetHealthProtocol` - Added `TargetDialer` struct which encapsulates existing TCP health check logic - Changed `Target` struct to use the `HealthChecker` interface - Changed `worker.checkHealth` to call the new `CheckHealth` function - Removed a `protocol` field from `healthCheckConfig` - Added `TargetHealthProtocolHTTP` for use with Kubernetes health checks - Moved and renamed test `Test_dialEndpoints` to `TestTargetDialer_dialEndpoints` - Added files `net.go` and `net_test.go` for `TargetDialer` Part of #58413 * Add health checks to Kubernetes agent and proxy (#59565) Implements core health check logic for Kubernetes. - Added functions `CheckHealth` and `GetProtocol` to `kubeDetails` - Added a `HealthCheckManager` field to the Kubernetes `TLSServerConfig` - Kube agent and proxy now instantiate a `HealthCheckManager` - Added a `HealthCheckConfigReader` to interfaces `ReadKubernetesAccessPoint` and `ReadProxyAccessPoint` - Added `HealthCheckConfig` read-only permissions for proxy and kube - Added `HealthCheckConfig` watching for proxy and kube - Added functions to `KubeServer` interface and `KubernetesServerV3` struct: - `GetTargetHealth() TargetHealth` - `SetTargetHealth(h TargetHealth)` - `GetTargetHealthStatus() TargetHealthStatus` - `SetTargetHealthStatus(status TargetHealthStatus)` - Health check supports dynamic discovery of Kubernetes clusters. Calls to `startHealthCheck()` and `stopHealthCheck()` were rearranged. - Added functions `startHeartbeatAndHealthCheck()` and `stopHeartbeatAndHealthCheck()` - Moved call to `HealthCheckManager.Start()` outside of the Kubernetes proxy server providing a future option to reuse `HealthCheckManager` in multiple proxy services - Removed `Status` initialization in KubernetesServerV3 `CheckAndSetDefaults()` - Added `kubernetesLabelMatchers` to `healthCheckConfig` struct. Default presets are omitted until the entire kube health check is complete. - Added Kubernetes matcher checking to `ValidateHealthCheckConfig()` - Changed `ValidateHealthCheckConfig()` to allow zero total DB matchers and Kubernetes matchers. - Changed KubernetesServerV3 `GetTargetHealthStatus()` to return `TargetHealthStatusUnknown` instead of an empty string - Changed health check worker `getTargetHealthTimeout` default timeout to `4s` from `10s`. This potentially reduces the response time of the initial heartbeat polling call to `GetServerInfo()`. Part of #58413 Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com> * Add UI for Kubernetes health checks (#59929) Adds health status indicators for Kubernetes clusters on the Resources page. Unhealthy clusters are highlighted, and clicking them opens a side panel displaying server information. Changes include: - New `KubeServer` protobuf message and `ListKubernetesServers` RPC - Web and Connect API endpoints for fetching Kubernetes server data - Health status filtering in `matchAndFilterKubeClusters` - `TargetHealth` fields added to frontend/backend types - Updated `StatusInfo.tsx` to display `kube_cluster` data Part of #58413 Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Add health-based connection routing for Kubernetes (#60083) Kubernetes servers are grouped by health, and dialed in order of healthy, unknown, then unhealthy, with random shuffling within each group for load distribution. The grouping and shuffling is implemented generically in a separate iterator function `OrderByTargetHealthStatus()` for reuse and testability. Changes: - Added `healthcheck.OrderByTargetHealthStatus()` function with tests Part of #58413 Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Allow disabling health check configs (#60447) Health check configuration can be disabled and enabled through the matcher. Disabling the matcher disables health checks for this configuration's resources including databases, Kubernetes clusters, and any future resources. Changes: - Added `disabled` field to health check matcher proto - Updated matcher selection logic - Updated unit tests Part of #58413 * Add documentation for Kubernetes health checks (#60201) A new Kubernetes-specific health check page is added. The existing Kubernetes troubleshooting documentation page is updated with health check specific error resolutions. Part of #58413 Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> Co-authored-by: Gavin Frazar <gavin.frazar@goteleport.com> Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> * Enable health checks for Kubernetes with virtual defaults (#60544) Health checks are enabled for all Kubernetes clusters by default. A design of creating one health check config default per resource is implemented. The choice eases adoption of health checks, supports existing clusters that already have database health checks, and avoids migrating the backend database. A new Kubernetes-specific `default-kube` health check config is added. And a database-specific `default` health check config already exists, and is preserved. A virtual default design is implemented by returning health check configs from memory if they don't exist in the backend database. The approach has the benefit of not re-inserting default values to the backend after they're deleted, which a prior approach had. Virtual defaults are added at the local health check service level, and returned from functions `GetHealthCheckConfig` and `ListHealthCheckConfigs`. Virtual defaults may be written, updated, and deleted to and from the backend. While virtual defaults may be deleted, it has the net effect of resetting the config to default settings, and matching all resources of that type (db, kube). Virtual defaults are always returned from health check `get` and `list` functions. Changes: - Added `default-kube` health check config specific to Kubernetes only - Updated local service functions `GetHealthCheckConfig` and `ListHealthCheckConfigs` to return virtual defaults - Added unit tests - Updated health check documentation with `default-kube` and info about virtual defaults Part of #58413 Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com> * Add `Resources` function for v18 backport The generic `Resources` function is backported to the generic `Service` type. `Resources` is used in a related commit enabling health checks for Kubernetes (#60544). Part of #58413 * Fix test for terraform health check config (#60640) Filters virtual defaults to allow explicit config references. Part of #58413 --------- Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> Co-authored-by: Gavin Frazar <gavin.frazar@goteleport.com> Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com>
rhammonds-teleport
pushed a commit
that referenced
this pull request
Nov 6, 2025
…er` interface (#59396) The main intent of refactoring is to provide health check extensibility for Kubernetes while supporting the existing DB health checks. A `HealthChecker` interface is added to support the different health check approaches of DBs and Kubernetes. Existing DB TCP health check logic is moved to a new `TargetDialer` struct. Changes: - Added `HealthChecker` interface with two functions: - `CheckHealth(ctx context.Context) ([]string, error)` - `GetProtocol() types.TargetHealthProtocol` - Added `TargetDialer` struct which encapsulates existing TCP health check logic - Changed `Target` struct to use the `HealthChecker` interface - Changed `worker.checkHealth` to call the new `CheckHealth` function - Removed a `protocol` field from `healthCheckConfig` - Added `TargetHealthProtocolHTTP` for use with Kubernetes health checks - Moved and renamed test `Test_dialEndpoints` to `TestTargetDialer_dialEndpoints` - Added files `net.go` and `net_test.go` for `TargetDialer` Part of #58413
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is part of Kubernetes health check integration.
The main intent of refactoring is to provide health check extensibility for Kubernetes while supporting the existing DB health checks. A
HealthCheckerinterface is added to support the different health check approaches of DBs and Kubernetes. Existing DB TCP health check logic is moved to a newTargetDialerstruct.In this PR:
HealthCheckerinterface with two functions:CheckHealth(ctx context.Context) ([]string, error)GetProtocol() types.TargetHealthProtocolTargetDialerstruct which encapsulates existing TCP health check logicTargetstruct to use theHealthCheckerinterfaceworker.checkHealthto call the newCheckHealthfunctionprotocolfield fromhealthCheckConfigTargetHealthProtocolHTTPfor use with Kubernetes health checksTest_dialEndpointstoTestTargetDialer_dialEndpointsnet.goandnet_test.goforTargetDialerRelates to: