On documenting sealed secrets usage in README (and its proper usage)

[sebiklamar]

Hi Vegard,

I already outlined in another issue (#106 (comment)) that I'd like to contribute to the READMEs. One area where I'm lacking knowledge is sealed secrets, hence the f. 2 questions:

1. Bootstrap: How do you create your sealed secrets initially? Please see some snippets I've gathered so far. Maybe you have some other commands handy -- maybe more elegant than my approach?
2. Regarding proper usage: It looks like you prefer storing the secrets in VCS permanently. That helps when in the need for re-creating the cluster (may it be for trial-and-error start from scratch or in case of disaster recovery). I will prepare a PR for storing the openssl key+crt in a SOPS-proteced file. So one doesn't depend on storing the openssl key locally (without VCS or backup). I'm contemplating this approach for terragrunt, at least, because using local files (not stored in VCS) is more tricky. What do you think?
3. How do you facilitate key rotation (mentioned in one of your blog articles)? As far as I understood the documentation, one could also create the K8S secrets first and then let the Controller generate/update the sealed secret upon. That would mean one could just assing a new ssl key to the controller and let K8S update all sealed secrets without the need to update the secrets

General approach

Do you create the sealed secret first which then creates a K8s secret or the other way round. Documentation for sealed secrets mentions both approaches. I'm using the generic pipeline approach based on kubeseal:

echo -n bar | kubectl create secret generic mysecret --dry-run=client --from-file=foo=/dev/stdin -o yaml | kubeseal --controller-namespace sealed-secrets -o yaml -n <namespace> --merge-into <existing or new sealed secrets>.yaml

CloudFlare API Token

I first created a local-managed CF tunnel and then piped it into kubectl-kubeseal chain:

cat ~/.cloudflared/<my-tunnel-id>.json | kubectl create secret generic tunnel-credentials --dry-run=client --from-file=credentials.json=/dev/stdin -o yaml | kubeseal --controller-namespace sealed-secrets -o yaml -n cloudflared --merge-into ~/src/vehagn-homelab/k8s/infra/network/cloudflared/tunnel-credentials.yaml

AdGuard

I just leveraged the users.yaml from another instance:

cat users.yaml | kubectl create secret generic users --dry-run=client --from-file=users.yaml=/dev/stdin -o yaml | kubeseal --controller-namespace sealed-secrets -o yaml -n dns --merge-into infra/network/dns/adguard/secret-users.yaml

Kind regards -- Sebastian

[vehagn]

Bootstrap: How do you create your sealed secrets initially?

It should be explained in this section.
I'm running

openssl req -x509 -days 365 -nodes -newkey rsa:4096 -keyout sealed-secrets.key -out sealed-secrets.cert -subj "/CN=sealed-secret/O=sealed-secret"

to create the certificate.

It looks like you prefer storing the secrets in VCS permanently

That's correct. I'll be the first to admit it's not the best way of doing it, but it's practical! I also store some of the secrets in clear-text in my local copy of this repo, though making sure I don't accidentally commit them.

How do you facilitate key rotation

Short answer; I don't. Again, this is not how it should be done! Without reading the whole thing, I think this is what you're looking for.

Do you create the sealed secret first which then creates a K8s secret or the other way round.

I use the kubernetes provider with this config.

CloudFlare API Token

Manually create a Secret

apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-token
  namespace: cert-manager
stringData:
  api-token: 'SECRET_API_TOKEN'

and run

kubeseal -o yaml --controller-namespace=sealed-secrets < secret-cert-manager-cloudflare-api.yaml > k8s/infra/controllers/cert-manager/cloudflare-api-token.yaml

AdGuard

apiVersion: v1
kind: Secret
metadata:
  name: users
  namespace: dns
data:
  users.yaml: "BASE64_ENCODED_STRING"

The BASE64_ENCODED_STRING looks like

users:
  - name: admin
    password: secret_passw0rd

And then a similar command as with the CF API token.

To summarise: I've made some security sacrifices for the sake of convenience. I have a note to address these in the future™.