Skip to content

[v13] Document discovery_group parameter #24712

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tigrato merged 1 commit into from
Apr 18, 2023
Merged

[v13] Document discovery_group parameter #24712

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs/pages/includes/discovery/discovery-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
discovery_service:
enabled: "yes"
# discovery_group is used to group discovered resources into different
# sets. This is useful when you have multiple Teleport Discovery services
# running in the same cluster but polling different cloud providers or cloud
# accounts. It prevents discovered services from colliding in Teleport when
# managing discovered resources.
discovery_group: "disc-group"
aws:
# AWS resource types. Valid options are:
# ec2 - discovers and registers AWS EC2 instances
Expand All @@ -11,7 +17,7 @@ discovery_service:
# Optional section: Defaults to "*":"*"
tags:
"*": "*"
# Optional section: install is used to provide parameters to the AWS SSM document.
# Optional section: install is used to provide parameters to the AWS SSM document.
# If the install section isn't provided, the below defaults are used.
# Only applicable for EC2 discovery.
install:
Expand Down
50 changes: 29 additions & 21 deletions docs/pages/kubernetes-access/discovery.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ description: Detailed guides for configuring Kubernetes Auto-Discovery.
layout: tocless-doc
---

Kubernetes Auto-Discovery allows Kubernetes clusters
hosted on cloud providers to be discovered and enrolled automatically.
Kubernetes Auto-Discovery allows Kubernetes clusters
hosted on cloud providers to be discovered and enrolled automatically.

While discovering a new Kubernetes cluster, Teleport does not install any component
While discovering a new Kubernetes cluster, Teleport does not install any component
on the cluster. Instead, it requires direct access to the cluster's API and
minimal access permissions.

Expand All @@ -24,43 +24,51 @@ Kubernetes Auto-Discovery consists of two steps:

### Polling cloud APIs

The Teleport Discovery Service is responsible for scanning the configured cloud
providers and identifying if any Kubernetes clusters match a set of filtering labels.
When the process identifies a new Kubernetes cluster, it creates a dynamic
resource within Teleport. This resource includes information imported from the
The Teleport Discovery Service is responsible for scanning the configured cloud
providers and identifying if any Kubernetes clusters match a set of filtering labels.
When the process identifies a new Kubernetes cluster, it creates a dynamic
resource within Teleport. This resource includes information imported from the
cloud provider such as:

- *Name*: Cluster name
- *Name*: Cluster name
- *Labels*
- Cluster tags.
- Cluster location.
- Identification of which cloud account the cluster belongs to — AWS Account ID / Azure Subscription ID.

<Notice type="tip">
You can import the cluster under a different name into Teleport's registry.
You can import the cluster under a different name into Teleport's registry.
To achieve this, you must attach the following tag to the resources — EKS and AKS — in your cloud provider:
- ***key***: `teleport.dev/kubernetes-name`
- ***value***: desired name
- ***key***: `teleport.dev/kubernetes-name`
- ***value***: desired name

The Discovery Service will check if the cluster includes the tag and use its value
The Discovery Service will check if the cluster includes the tag and use its value
as the resource name in Teleport.

You should use this feature whenever there are clusters in different regions/cloud providers
You should use this feature whenever there are clusters in different regions/cloud providers
with the same name to prevent them from colliding in Teleport.

</Notice>

In addition to detecting new Kubernetes clusters, the Discovery Service also removes
— from Teleport's registry — the Kubernetes clusters that have been deleted or whose tags
In addition to detecting new Kubernetes clusters, the Discovery Service also removes
— from Teleport's registry — the Kubernetes clusters that have been deleted or whose tags
no longer meet the filtering labels.

The following snippet describes the different configuration options for the Discovery Service
(!docs/pages/kubernetes-access/discovery/includes/discovery-group.mdx!)

The following snippet describes the different configuration options for the Discovery Service
and their default values.

```yaml
# This section configures the Discovery Service
discovery_service:
enabled: "yes"
# discovery_group is used to group discovered resources into different
# sets. This is useful when you have multiple Teleport Discovery services
# running in the same cluster but polling different cloud providers or cloud
# accounts. It prevents discovered services from colliding in Teleport when
# managing discovered resources.
discovery_group: "prod"
aws:
# AWS resource types. Valid options are:
# eks - discovers and registers AWS EKS clusters
Expand Down Expand Up @@ -97,12 +105,12 @@ discovery_service:

### Forwarding requests to the Kubernetes Cluster

The Teleport Kubernetes Service is responsible for monitoring the dynamic resources created or
The Teleport Kubernetes Service is responsible for monitoring the dynamic resources created or
updated by the Discovery Service and forwarding requests to the Kubernetes clusters they represent.
To work correctly, it requires direct access to the target Kubernetes clusters and
To work correctly, it requires direct access to the target Kubernetes clusters and
permissions to forward requests.

To turn on dynamic resource monitoring in the Kubernetes Service, you must configure
To turn on dynamic resource monitoring in the Kubernetes Service, you must configure
the `kubernetes_service.resources` section as shown in the following snippet:

```yaml
Expand All @@ -115,5 +123,5 @@ kubernetes_service:
"*": "*" # can be configured to limit the clusters to watched by this service.
```

Both services — Discovery and Kubernetes — can be configured in the same
Teleport process or separate processes.
Both services — Discovery and Kubernetes — can be configured in the same
Teleport process or separate processes.
125 changes: 64 additions & 61 deletions docs/pages/kubernetes-access/discovery/aws.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ title: Teleport EKS Auto-Discovery (Preview)
description: How to configure auto-discovery of AWS EKS clusters in Teleport.
---

EKS Auto-Discovery can automatically
discover any EKS cluster and enroll it in Teleport if its tags match the
configured labels.
EKS Auto-Discovery can automatically
discover any EKS cluster and enroll it in Teleport if its tags match the
configured labels.

(!docs/pages/kubernetes-access/discovery/includes/step-description.mdx!)

Expand All @@ -16,28 +16,28 @@ configured labels.
- An AWS account with permissions to create and attach IAM policies.
- An AWS account with `system:masters` RBAC access to EKS clusters.
- A host to run the Teleport Discovery and Kubernetes services.
- One or more EKS clusters running.
- One or more EKS clusters running.

## Known limitations

Due to the authorization method that AWS has chosen for EKS clusters, it is not
possible for the Teleport process to configure the necessary permissions to forward
Due to the authorization method that AWS has chosen for EKS clusters, it is not
possible for the Teleport process to configure the necessary permissions to forward
requests to EKS clusters.
This limitation exists because the authorization method on EKS clusters requires that the IAM role -
used for authentication - exists in the
[`aws-auth`](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html) `ConfigMap`.
used for authentication - exists in the
[`aws-auth`](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html) `ConfigMap`.

In this `ConfigMap`, IAM roles are mapped to Kubernetes RBAC users or groups that are then
used by Kubernetes RBAC to grant access permissions. If the mapping does not exist, it results in
In this `ConfigMap`, IAM roles are mapped to Kubernetes RBAC users or groups that are then
used by Kubernetes RBAC to grant access permissions. If the mapping does not exist, it results in
unauthorized requests. Therefore, Teleport cannot edit the `ConfigMap`
without first having access to the cluster.

If Teleport is not running with the same IAM identity that creates all clusters,
please ensure you configure the required permissions as described in Step 2.
If Teleport is not running with the same IAM identity that creates all clusters,
please ensure you configure the required permissions as described in Step 2.

<Admonition type="note">

The AWS EKS team is working on a feature request to introduce an external API to manage access to the cluster without manually editing the Configmap ([aws/containers-roadmap#185](https://github.com/aws/containers-roadmap/issues/185)).
The AWS EKS team is working on a feature request to introduce an external API to manage access to the cluster without manually editing the Configmap ([aws/containers-roadmap#185](https://github.com/aws/containers-roadmap/issues/185)).

Hopefully, once the feature is available, Teleport can leverage it to configure its access to the cluster.

Expand All @@ -46,7 +46,7 @@ Hopefully, once the feature is available, Teleport can leverage it to configure
## Step 1/3. Set up AWS IAM credentials

For Teleport to discover EKS clusters, the instance
running the Discovery Service requires an IAM policy that allows Teleport to list and
running the Discovery Service requires an IAM policy that allows Teleport to list and
describe EKS clusters:

```js
Expand All @@ -62,39 +62,39 @@ describe EKS clusters:
"Resource": "*"
}
]
}
}
```

## Step 2/3. Configure EKS cluster authorization

When the Kubernetes Service uses an IAM role that is different from the one that
creates the clusters, you need to configure the mapping between the Teleport IAM
Role and the Kubernetes RBAC permissions by editing the `aws-auth` `Configmap` on
each of the discovered clusters.
When the Kubernetes Service uses an IAM role that is different from the one that
creates the clusters, you need to configure the mapping between the Teleport IAM
Role and the Kubernetes RBAC permissions by editing the `aws-auth` `Configmap` on
each of the discovered clusters.

To forward requests to the Kubernetes cluster, the Teleport Kubernetes Service
requires cluster-wide permissions to `Impersonate` RBAC users and groups, to
create `SelfSubjectAccessReviews` and `SelfSubjectRulesReviews`, and, finally,
To forward requests to the Kubernetes cluster, the Teleport Kubernetes Service
requires cluster-wide permissions to `Impersonate` RBAC users and groups, to
create `SelfSubjectAccessReviews` and `SelfSubjectRulesReviews`, and, finally,
read access to `Pods`.

If your Kubernetes cluster does not have an RBAC group with the required
permissions, you can create the `ClusterRole`, `ClusterRoleBinding`, and the
mapping by following the Creating RBAC group guide.
If your cluster already has an RBAC group that satisfies the required permissions,
you can reuse it and map it into the IAM Role used by the Teleport Kubernetes
Service. For simplicity, it is also possible to map the Teleport IAM role onto
a built-in Kubernetes RBAC group like `system:masters`, but not recommended in
If your Kubernetes cluster does not have an RBAC group with the required
permissions, you can create the `ClusterRole`, `ClusterRoleBinding`, and the
mapping by following the Creating RBAC group guide.
If your cluster already has an RBAC group that satisfies the required permissions,
you can reuse it and map it into the IAM Role used by the Teleport Kubernetes
Service. For simplicity, it is also possible to map the Teleport IAM role onto
a built-in Kubernetes RBAC group like `system:masters`, but not recommended in
production.

<Tabs>
<TabItem label="Creating RBAC group" >

Connect to your target cluster with your credentials and create the following
Connect to your target cluster with your credentials and create the following
resources using `kubectl`.

### ClusterRole

Create the `ClusterRole` RBAC definition with the required permissions for Teleport
Create the `ClusterRole` RBAC definition with the required permissions for Teleport
Kubernetes Service to forward requests to the cluster.

```yaml
Expand Down Expand Up @@ -147,9 +147,9 @@ subjects:

### IAM mapping

Finally, edit the `configmap/aws-auth` in the `kube-system` namespace and append
the following to `mapRoles`. Replace `{teleport_aws_iam_role}` with the
appropriate IAM role that Teleport Kubernetes Service will use.
Finally, edit the `configmap/aws-auth` in the `kube-system` namespace and append
the following to `mapRoles`. Replace `{teleport_aws_iam_role}` with the
appropriate IAM role that Teleport Kubernetes Service will use.
This step will link the Teleport IAM role into the Kubernetes RBAC group `teleport`,
allowing Teleport Kubernetes Service to forward requests into the cluster.

Expand All @@ -163,15 +163,15 @@ data:
username: teleport
```

At this point, the Teleport IAM role already has the minimum permissions
to forward requests to the cluster.
At this point, the Teleport IAM role already has the minimum permissions
to forward requests to the cluster.

</TabItem>
<TabItem label="Reuse an existing RBAC Group" >

To associate the Teleport IAM role with an existing Kubernetes RBAC group,
edit the `configmap/aws-auth` in the `kube-system` namespace and append
the following to `mapRoles`.
To associate the Teleport IAM role with an existing Kubernetes RBAC group,
edit the `configmap/aws-auth` in the `kube-system` namespace and append
the following to `mapRoles`.

```yaml
apiVersion: v1
Expand All @@ -184,26 +184,26 @@ data:
username: teleport
```

Please replace `{teleport_aws_iam_role}` with the appropriate IAM role that
Teleport Kubernetes Service is using and `{rbac_group}` with the existing Kubernetes
Please replace `{teleport_aws_iam_role}` with the appropriate IAM role that
Teleport Kubernetes Service is using and `{rbac_group}` with the existing Kubernetes
RBAC Group that satisfies the required permissions.

At this point, the Teleport IAM role already has the minimum permissions
to forward requests to the cluster.
At this point, the Teleport IAM role already has the minimum permissions
to forward requests to the cluster.

</TabItem>

<TabItem label="Use system:masters" >

<Notice type="warning">
Granting the `system:masters` group to the IAM role associated with the Teleport
service means granting administrator-level permissions on the Kubernetes cluster.
Granting the `system:masters` group to the IAM role associated with the Teleport
service means granting administrator-level permissions on the Kubernetes cluster.
To follow least privilege principle we do not recommend using this method in production.
</Notice>

To associate the Teleport IAM role with the `system:masters` RBAC group,
edit the `configmap/aws-auth` in the `kube-system` namespace and append
the following to `mapRoles`.
To associate the Teleport IAM role with the `system:masters` RBAC group,
edit the `configmap/aws-auth` in the `kube-system` namespace and append
the following to `mapRoles`.

```yaml
apiVersion: v1
Expand All @@ -216,28 +216,28 @@ data:
username: teleport
```

Please replace `{teleport_aws_iam_role}` with the appropriate IAM role that
Please replace `{teleport_aws_iam_role}` with the appropriate IAM role that
Teleport Kubernetes Service is using.

At this point, the Teleport IAM role already has the minimum permissions
to forward requests to the cluster.
At this point, the Teleport IAM role already has the minimum permissions
to forward requests to the cluster.

</TabItem>
</Tabs>

<Notice type="tip">
If you provision your EKS clusters using tools such as `terraform`, `eksctl` or
`Cloudformation`, you can use them to automatically configure the `aws-auth` `Configmap`
If you provision your EKS clusters using tools such as `terraform`, `eksctl` or
`Cloudformation`, you can use them to automatically configure the `aws-auth` `Configmap`
and create the `ClusterRole` and `ClusterRoleBinding` resources during cluster provisioning.
</Notice>

## Step 3/3. Configure Teleport to discover EKS clusters

### Get a join token

Teleport EKS Auto-Discovery requires a valid Teleport auth token for the Discovery and
Kubernetes services to join the cluster. Generate one by running the following
command against your Teleport Auth Service and save it in `/tmp/token` on the
Teleport EKS Auto-Discovery requires a valid Teleport auth token for the Discovery and
Kubernetes services to join the cluster. Generate one by running the following
command against your Teleport Auth Service and save it in `/tmp/token` on the
machine that will run Kubernetes Auto-Discovery:

```code
Expand All @@ -246,11 +246,13 @@ $ tctl tokens add --type=discovery,kube

### Configure the Teleport Kubernetes and Discovery Services

Enabling EKS Auto-Discovery requires that the `discovery_service.aws` section
include at least one entry and that `discovery_service.aws.types` include `eks`.
It also requires configuring the `kubernetes_service.resources.tags` to use the same
labels configured at `discovery_service.aws.tags` or a subset of them to make
the Kubernetes Service listen to the dynamic resources created by the Discovery
(!docs/pages/kubernetes-access/discovery/includes/discovery-group.mdx!)

Enabling EKS Auto-Discovery requires that the `discovery_service.aws` section
include at least one entry and that `discovery_service.aws.types` include `eks`.
It also requires configuring the `kubernetes_service.resources.tags` to use the same
labels configured at `discovery_service.aws.tags` or a subset of them to make
the Kubernetes Service listen to the dynamic resources created by the Discovery
Service.

```yaml
Expand All @@ -269,6 +271,7 @@ ssh_service:
enabled: off
discovery_service:
enabled: "yes"
discovery_group: "aws-prod"
aws:
- types: ["eks"]
regions: ["*"]
Expand Down
Loading