Skip to content

Conversation

@marun
Copy link
Contributor

@marun marun commented Dec 11, 2019

@openshift-ci-robot openshift-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Dec 11, 2019
@marun marun changed the title Enable Bound Service Account Tokens WIP Enable Bound Service Account Tokens Dec 11, 2019
@openshift-ci-robot openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 11, 2019

### Goals

- A pod can request a bound service account token via a projected volume.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

numbered list please

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


### Non-Goals

- Integrate or ship the aws pod identity webhook
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@derekwaynecarr @cuppett do we get to punt or not?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the epic this is a non-goal for this team and this release.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was under the assumption that one of the exit criteria would be to make sure the iam setup works.. I don't think we have to ship it, but we have to verify it works with sane steps.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is called out in the testing section.

in kube > 1.12. They are only configured by the apiserver if the following options are
provided:
- service-account-signing-key-file (set to private key managed by operator)
- service-account-issuer (defaults to https://kubernetes.default.svc/)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we default to something more specific/unique per cluster (thinking a derivative of the cluster ID)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that the default specified here is the same as the upstream default. What would be the use case for providing something unique?

@marun marun changed the title WIP Enable Bound Service Account Tokens Enable Bound Service Account Tokens Jan 4, 2020
@openshift-ci-robot openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 4, 2020

## Summary

Enable the optional use of bound service account tokens via volume projection and the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

links to the APIs would be nice.

Copy link
Contributor Author

@marun marun Jan 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Note that the best doc on both of these is arguably the jpweber.io link in see-also

bound tokens.
- The keypair should be written to a secret in the `openshift-kube-apiserver` namespace
- The public key should be added to (rather than replaced in) a configmap in the
`openshift-config-managed` namespace.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it surprises me that there is no standard location defined by upstream.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it surprises me that there is no standard location defined by upstream.

our operator configuration uses this standard location. Recall that the kube-apiserver doesn't read its config from in-cluster sources.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meant the public key. I guess the kube-apiserver is the only one verifying that, so it is the only one that has to know its location.

`KubeAPIServerConfig.ServiceAccountPublicKeyFiles` for provision to the apiserver.
- In the event that the keypair secret is deleted, a new keypair will be generated
but tokens signed by the previous private key will continue to validate since the
corresponding public key will still be used to validate bound tokens.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so the openshift-config-managed configmap has multiple public key possibly? How are they named?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to reflect: sequentially, in the same manner as sa token public keys are maintained.


It probably makes sense to manually verify integration with the AWS pod identity webhook,
since this is the primary motivation for this enhancement. It may make sense to delegate
this testing to those that will be deploying the webhook for customers.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe deletion of the secret would make sense to test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to include testing of both secret and configmap removal.

@deads2k
Copy link
Contributor

deads2k commented Jan 9, 2020

/approve

@sttts has lgtm

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 9, 2020
- Operator should set this to the path of the private key it manages
- service-account-issuer
- Operator should set this to https://kubernetes.default.svc/
- Operator should default this to `bound-sa-token.auth.openshift.io` and it should be
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is changed, which consequences does it have (especially for existing tokens) ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Existing tokens signed by the previous issuer will fail to validate. If long-lived tokens have been issued to pods, pod restart will be the only fix.

- Validating that a bound token can be requested via the TokenRequest API
- Validating that a pod can request a bound token via volume projection

The key management proposed by this enhancement will also need to tested:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: be tested

- Would it make sense to disable bound tokens if --service-account-issuer is not set? Or
should the feature be enabled by default with iss defaulted if not provided on initial
deployment, and it could be changed from the default subsequently if required?
- How does the kubelet respond to a change in issuer?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still open?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which of the two will be implemented now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--service-account-issuer will be defaulted to allow the feature to come enabled out of the box.

@marun
Copy link
Contributor Author

marun commented Jan 21, 2020

@sttts Updated, PTAL

- Operator should set this to the path of the private key it manages
- service-account-issuer
- Operator should default this to `auth.openshift.io` and it should be possible to
override it. When it is overriden, tokens from the previous issuer will no longer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: overridden

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

- Enabling bound tokens will satisfy goals #1 and #2.

- The apiserver operator should manage a keypair used to sign and verify bound tokens
- An rsa keypair will be used. The trust model for digital signatures (via jwt) is
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: RSA

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


### Implementation Details/Notes/Constraints [optional]

Bound tokens are not currently required until bootstrap is complete, so it should be
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @bergerhoffer, this needs to go into docs as a warning

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sttts I'll make a note of this.

@sttts
Copy link
Contributor

sttts commented Jan 21, 2020

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 21, 2020
@openshift-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, marun, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants