Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

infra/gcp: manage aaa secrets via terraform #3028

Merged
merged 1 commit into from
Nov 3, 2021

Conversation

spiffxp
Copy link
Member

@spiffxp spiffxp commented Nov 3, 2021

Related:

This removes the last of the bash-based secret provisioning code. This will require terraform import commands to deploy

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. area/bash Bash scripts, testing them, writing less of them, code in infra/gcp/ area/infra Infrastructure management, infrastructure design, code in infra/ approved Indicates a PR has been approved by an approver from all required OWNERS files. area/terraform Terraform modules, testing them, writing more of them, code in infra/gcp/clusters/ labels Nov 3, 2021
@k8s-ci-robot k8s-ci-robot added the sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. label Nov 3, 2021
@spiffxp
Copy link
Member Author

spiffxp commented Nov 3, 2021

/cc @ameukam

@spiffxp
Copy link
Member Author

spiffxp commented Nov 3, 2021

Perhaps unsurprisingly, the bash script to import all of this looks not unlike the bash script that is being removed...

#!/usr/bin/env bash
project="kubernetes-public"
secret_specs=()
prow_secrets=(
    k8s-infra-build-clusters-kubeconfig
    k8s-infra-cherrypick-robot-github-token
    k8s-infra-ci-robot-github-account-password
    k8s-infra-ci-robot-github-token
    k8s-infra-prow-cookie
    k8s-infra-prow-github-oauth-config
    k8s-infra-prow-hmac-token
)
publishing_bot_secrets=(
    publishing-bot-github-token
)
slack_infra_secrets=(
    recaptcha-secret-key
    recaptcha-site-key
    slack-event-log-config
    slack-moderator-config
    slack-moderator-words-config
    slack-post-message-config
    slack-welcomer-config
    slackin-token
)
triageparty_release_secrets=(
    triage-party-github-token
)
elekto_secrets=(
    elekto-db-database
    elekto-db-host
    elekto-db-password
    elekto-db-port
    elekto-db-username
    elekto-github-client-id
    elekto-github-client-secret
    elekto-meta-secret
)
mapfile -t secret_specs < <(
    printf "%s/prow\n" "${prow_secrets[@]}"
    printf "%s/publishing-bot\n" "${publishing_bot_secrets[@]}"
    printf "%s/slack-infra\n" "${slack_infra_secrets[@]}"
    printf "%s/triageparty-release\n" "${triageparty_release_secrets[@]}"
    printf "%s/elekto\n" "${elekto_secrets[@]}"
)

for spec in "${secret_specs[@]}"; do
    secret="$(echo "${spec}" | cut -d/ -f1)"
    app="$(echo "${spec}" | cut -d/ -f2)"

    owners="k8s-infra-rbac-${app}@kubernetes.io"
    role="roles/secretmanager.admin"
    
    terraform import \
      google_secret_manager_secret.aaa_app_secrets[\"${secret}\"] \
      "projects/${project}/secrets/${secret}"

    terraform import \
      google_secret_manager_secret_iam_binding.aaa_app_secret_admins[\"${secret}\"] \
      "projects/${project}/secrets/${secret} ${role} group:${owners}"
done

Verifying via

terraform workspace select default
terraform state pull > terraform.tfstate
terraform workspace select spiffxp
terraform state push terraform.tfstate
terraform plan # lots of stuff will be changed
./update-state.sh
terraform plan # nothing to change

@spiffxp
Copy link
Member Author

spiffxp commented Nov 3, 2021

/hold
Some - vs _ typos

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 3, 2021
@spiffxp
Copy link
Member Author

spiffxp commented Nov 3, 2021

This will result in one actual change which I think is fine given that we've got this password in 1password now

  # google_secret_manager_secret_iam_binding.aaa_app_secret_admins["k8s-infra-ci-robot-github-account-password"] will be updated in-place
  ~ resource "google_secret_manager_secret_iam_binding" "aaa_app_secret_admins" {
        id        = "projects/kubernetes-public/secrets/k8s-infra-ci-robot-github-account-password/roles/secretmanager.admin"
      ~ members   = [
          - "group:[email protected]",
            # (1 unchanged element hidden)
        ]
        # (4 unchanged attributes hidden)
    }

@spiffxp
Copy link
Member Author

spiffxp commented Nov 3, 2021

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 3, 2021
Copy link
Member

@endocrimes endocrimes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/hold

don't want to cause this to merge immediately bc I'm still new to k8s-infra, I've validated that all of the secrets should align the same way, and should evaluate equivalently (plus also saw @spiffxp's testing). The DSL is also fairly nice for managing secrets in the future.

]
},
}
// Even though we could just use the list, we're going to keep parity with
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

honestly also makes for a fairly nice DSL

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 3, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: endocrimes, spiffxp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 3, 2021
@ameukam
Copy link
Member

ameukam commented Nov 3, 2021

/lgtm

@spiffxp
Copy link
Member Author

spiffxp commented Nov 3, 2021

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 3, 2021
@k8s-ci-robot k8s-ci-robot merged commit 7124660 into kubernetes:main Nov 3, 2021
@k8s-ci-robot k8s-ci-robot added this to the v1.23 milestone Nov 3, 2021
@spiffxp spiffxp deleted the aaa-secrets-terraform branch November 3, 2021 20:32
@spiffxp
Copy link
Member Author

spiffxp commented Nov 3, 2021

Deployed by updating state using script above and running terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # google_secret_manager_secret_iam_binding.aaa_app_secret_admins["k8s-infra-ci-robot-github-account-password"] will be updated in-place
  ~ resource "google_secret_manager_secret_iam_binding" "aaa_app_secret_admins" {
        id        = "projects/kubernetes-public/secrets/k8s-infra-ci-robot-github-account-password/roles/secretmanager.admin"
      ~ members   = [
          - "group:[email protected]",
            # (1 unchanged element hidden)
        ]
        # (4 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.
╷
│ Warning: Deprecated Attribute
│
│   with google_container_cluster.cluster,
│   on 10-cluster-configuration.tf line 78, in resource "google_container_cluster" "cluster":
│   78: resource "google_container_cluster" "cluster" {
│
│ Basic authentication was removed for GKE cluster versions >= 1.19.
│
│ (and 2 more similar warnings elsewhere)
╵

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

google_secret_manager_secret_iam_binding.aaa_app_secret_admins["k8s-infra-ci-robot-github-account-password"]: Modifying... [id=projects/kubernetes-public/secrets/k8s-infra-ci-robot-github-account-password/roles/secretmanager.admin]
google_secret_manager_secret_iam_binding.aaa_app_secret_admins["k8s-infra-ci-robot-github-account-password"]: Modifications complete after 4s [id=projects/kubernetes-public/secrets/k8s-infra-ci-robot-github-account-password/roles/secretmanager.admin]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/bash Bash scripts, testing them, writing less of them, code in infra/gcp/ area/infra Infrastructure management, infrastructure design, code in infra/ area/terraform Terraform modules, testing them, writing more of them, code in infra/gcp/clusters/ cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants