Skip to content

Kubernetes joinMethod#18659

Merged
hugoShaka merged 6 commits intomasterfrom
hugo/kube-join-tokens
Dec 2, 2022
Merged

Kubernetes joinMethod#18659
hugoShaka merged 6 commits intomasterfrom
hugo/kube-join-tokens

Conversation

@hugoShaka
Copy link
Copy Markdown
Contributor

This PR adds a new joinMethod as described in #17905

This method allow pods running in the same Kubernetes cluster than the auth servers to join the Teleport cluster. It relies on Kubernetes tokens to establish trust. The goal is to be able to deploy proxies and auths separately and join them in a single cluser.

Pre Kubernetes 1.20, the tokens are static, long-lived, not bound to pods. We support them for compatibility reasons. Starting with Kubernetes 1.20, tokens are bound to pods (and starting with 1.21 they can be mounted through projected volumes). Starting with 1.21 we should only accept bound tokens. The chart will ensure tokens are properly mounted with projected volumes so we can benefit from the 1h to 10min token lifetime.

@hugoShaka hugoShaka requested a review from strideynet November 21, 2022 21:20
Copy link
Copy Markdown
Contributor

@strideynet strideynet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good - nice work. Just a few nits.

I couldn't leave this as a comment, but a lib/kubernetestoken/kubernetestoken.go which just has a comment linking to the RFD and also linking to the appropriate docs on the Kubernetes site would be 👌

I'm thinking we maybe ought to group all of the join method packages at some-point now that we have 3 ! Perhaps that's better suited for a separate PR though...

Comment thread api/types/provisioning.go Outdated
Comment thread lib/auth/auth.go Outdated
Comment thread lib/auth/join_kubernetes.go Outdated
Comment thread lib/kubernetestoken/token_validator.go Outdated
Comment thread lib/kubernetestoken/token_validator.go Outdated
Comment thread lib/kubernetestoken/token_validator.go Outdated
Comment thread lib/kubernetestoken/token_validator_test.go Outdated
Comment thread lib/kubernetestoken/token_source.go Outdated
Copy link
Copy Markdown

@phosphore phosphore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This (#18659) & #17905 LGTM on a high level!

Comment thread lib/kubernetestoken/token_validator.go Outdated
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While the prefix system: should be reserved for internal Kubernetes system use, I believe nothing prevents a normal user or group to start with the system: prefix and bypass this? https://kubernetes.io/docs/reference/access-authn-authz/rbac/#referring-to-subjects
A better check would probably involve looking at the assigned groups (i.e. at least reviewResult.Status.User.Groups including system:serviceaccounts).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was not able to find any information about checks and security enforcement around username prefix and reserved groups. I will add the group check as well.

@hugoShaka hugoShaka changed the title Hugo/kube join tokens Kubernetes joinMethod Nov 23, 2022
@hugoShaka hugoShaka force-pushed the hugo/kube-join-tokens branch 2 times, most recently from 28cd8d5 to da20949 Compare November 29, 2022 16:53
@hugoShaka hugoShaka marked this pull request as ready for review November 29, 2022 16:53
Comment thread api/types/provisioning.go Outdated
Comment thread api/types/provisioning.go
Comment thread lib/kubernetestoken/token_validator.go Outdated
Comment thread lib/kubernetestoken/token_validator.go
Comment thread lib/auth/join_kubernetes.go Outdated
@hugoShaka hugoShaka requested a review from nklaassen December 1, 2022 15:11
@github-actions github-actions Bot removed the request for review from nklaassen December 1, 2022 15:11
Copy link
Copy Markdown
Contributor

@nklaassen nklaassen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this line could probably use an update

Do you think we should add an item to the test plan for a manual check of this in a real kube cluster?

It all looks pretty good to me, I left mostly minor comments you can take or leave or do in a separate PR.

Comment thread lib/auth/join_kubernetes.go Outdated
Comment thread lib/kubernetestoken/token_validator.go
Comment thread lib/kubernetestoken/token_validator.go Outdated
Comment thread lib/kubernetestoken/token_validator.go
Comment thread lib/kubernetestoken/token_validator.go
Comment thread api/proto/teleport/legacy/types/types.proto
Comment thread lib/kubernetestoken/token_source.go Outdated
Comment thread lib/auth/join_kubernetes.go
Comment thread lib/auth/join.go
@hugoShaka
Copy link
Copy Markdown
Contributor Author

hugoShaka commented Dec 2, 2022

Following a discussion with nic I will move tbot out of this PR scope as it is untested and was broken. I definitely will bring this feature back once we have a working tbot in kubernetes (like the operator), this will allow to move the teleport operator out if its sidecar and have it join the cluster nicely.

Comment thread api/proto/teleport/legacy/types/types.proto Outdated
Comment thread lib/auth/bot.go Outdated
Comment thread lib/auth/join.go Outdated
hugoShaka and others added 2 commits December 2, 2022 14:31
Co-authored-by: Noah Stride <noah.stride@goteleport.com>
Co-authored-by: Isaiah Becker-Mayer <isaiah@goteleport.com>
@hugoShaka hugoShaka force-pushed the hugo/kube-join-tokens branch from f89e6d0 to 21968ff Compare December 2, 2022 19:32
@hugoShaka hugoShaka force-pushed the hugo/kube-join-tokens branch from 6cc5ffd to 5da9d0d Compare December 2, 2022 19:57
Copy link
Copy Markdown
Contributor

@nklaassen nklaassen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@hugoShaka hugoShaka enabled auto-merge (squash) December 2, 2022 20:08
@hugoShaka hugoShaka merged commit f0dd7d7 into master Dec 2, 2022
@hugoShaka hugoShaka deleted the hugo/kube-join-tokens branch December 2, 2022 21:08
hugoShaka added a commit that referenced this pull request Feb 16, 2023
This commit adds a new joinMethod as described in #17905

This method allow pods running in the same Kubernetes cluster than the auth servers to join the Teleport cluster. It relies on Kubernetes tokens to establish trust. The goal is to be able to deploy proxies and auths separately and join them in a single cluser.

Pre Kubernetes 1.20, the tokens are static, long-lived, not bound to pods. We support them for compatibility reasons. Starting with Kubernetes 1.20, tokens are bound to pods (and starting with 1.21 they can be mounted through projected volumes). Starting with 1.21 we should only accept bound tokens. The chart will ensure tokens are properly mounted with projected volumes so we can benefit from the 1h to 10min token lifetime.
hugoShaka added a commit that referenced this pull request Feb 20, 2023
This commit adds a new joinMethod as described in #17905

This method allow pods running in the same Kubernetes cluster than the auth servers to join the Teleport cluster. It relies on Kubernetes tokens to establish trust. The goal is to be able to deploy proxies and auths separately and join them in a single cluser.

Pre Kubernetes 1.20, the tokens are static, long-lived, not bound to pods. We support them for compatibility reasons. Starting with Kubernetes 1.20, tokens are bound to pods (and starting with 1.21 they can be mounted through projected volumes). Starting with 1.21 we should only accept bound tokens. The chart will ensure tokens are properly mounted with projected volumes so we can benefit from the 1h to 10min token lifetime.
hugoShaka added a commit that referenced this pull request Feb 24, 2023
This commit adds a new joinMethod as described in #17905

This method allow pods running in the same Kubernetes cluster than the auth servers to join the Teleport cluster. It relies on Kubernetes tokens to establish trust. The goal is to be able to deploy proxies and auths separately and join them in a single cluser.

Pre Kubernetes 1.20, the tokens are static, long-lived, not bound to pods. We support them for compatibility reasons. Starting with Kubernetes 1.20, tokens are bound to pods (and starting with 1.21 they can be mounted through projected volumes). Starting with 1.21 we should only accept bound tokens. The chart will ensure tokens are properly mounted with projected volumes so we can benefit from the 1h to 10min token lifetime.
hugoShaka added a commit that referenced this pull request Feb 24, 2023
Backport #18659

### About the backport

This backport is here to provide a rollback path from v12 to v11. Helm users upgrading to v12 end up with Kubernetes tokens automatically created. Currently, if they rollback to v11, the token is unknown and Teleport crashes during cache warming. Once this will be backported, users will have a v11 version they can rollback to if the v12 upgrade fails.

### Original commit message

This commit adds a new joinMethod as described in #17905

This method allow pods running in the same Kubernetes cluster than the auth servers to join the Teleport cluster. It relies on Kubernetes tokens to establish trust. The goal is to be able to deploy proxies and auths separately and join them in a single cluser.

Pre Kubernetes 1.20, the tokens are static, long-lived, not bound to pods. We support them for compatibility reasons. Starting with Kubernetes 1.20, tokens are bound to pods (and starting with 1.21 they can be mounted through projected volumes). Starting with 1.21 we should only accept bound tokens. The chart will ensure tokens are properly mounted with projected volumes so we can benefit from the 1h to 10min token lifetime.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants