Add budget to k8s-infra-ii-sandbox

[ameukam]

Add a defined budget to k8s-infra-ii-sandbox but also use the project
to experiment GCP budgets.

Ref: #1375

Signed-off-by: Arnaud Meukam ameukam@gmail.com

[ameukam]

It's purely random. Not sure if there is a requirement for this.

[hh]

What are "units" ? How do we measure our usage of "units"?
:)

[ameukam]

Units represents the amount assigned to the project. It should be part of the public billing report once you start consume resources.

[ameukam]

/assign @spiffxp

[spiffxp]

/approve
/lgtm
/hold
when ready to deploy

[ameukam]

Edit: Look like I need the billing service and the role role/billing.admin before I run this :

Error: Error creating Budget: googleapi: Error 403: Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the billingbudgets.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "domain": "googleapis.com",
    "metadata": {
      "consumer": "projects/764086051850",
      "service": "billingbudgets.googleapis.com"
    },
    "reason": "SERVICE_DISABLED"
  }
]

[ameukam]

After some reading of the IAM roles for billing I'll need the role :

roles/billing.creator | Billing Account Creator | Provides access to create billing accounts. | billing.accounts.create resourcemanager.organizations.get | Organization

at the organization level.

I was expecting Org admins be able to do that but it's not the case.

gcloud iam roles describe organization.admin --organization=758905017065
description: Access to administer all resources belonging to the organization
etag: BwXAqL6HDQU=
includedPermissions:
- billing.accounts.get
- billing.accounts.getIamPolicy
- billing.accounts.list
- billing.accounts.redeemPromotion
- billing.credits.list
- billing.resourceAssociations.create
- orgpolicy.policy.get
- resourcemanager.folders.create
- resourcemanager.folders.delete
- resourcemanager.folders.get
- resourcemanager.folders.getIamPolicy
- resourcemanager.folders.list
- resourcemanager.folders.move
- resourcemanager.folders.setIamPolicy
- resourcemanager.folders.undelete
- resourcemanager.folders.update
- resourcemanager.organizations.get
- resourcemanager.organizations.getIamPolicy
- resourcemanager.organizations.setIamPolicy
- resourcemanager.projects.create
- resourcemanager.projects.get
- resourcemanager.projects.getIamPolicy
- resourcemanager.projects.list
- resourcemanager.projects.move
- resourcemanager.projects.setIamPolicy
name: organizations/758905017065/roles/organization.admin
stage: GA
title: Organization Admin

[spiffxp]

I'm re-running terraform apply for k8s-infra-ii-sandbox at HEAD to work on migrating to new gcs buckets as part of #1952

It looks like it's going to undo some partial changes deployed from this PR

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # google_monitoring_notification_channel.wg_k8s_infra_leads will be destroyed
  - resource "google_monitoring_notification_channel" "wg_k8s_infra_leads" {
      - display_name = "Notification Alert for Budget Threshold" -> null
      - enabled      = true -> null
      - id           = "projects/k8s-infra-ii-sandbox/notificationChannels/10299170612928618272" -> null
      - labels       = {
          - "email_address" = "wg-k8s-infra-leads@kubernetes.io"
        } -> null
      - name         = "projects/k8s-infra-ii-sandbox/notificationChannels/10299170612928618272" -> null
      - project      = "k8s-infra-ii-sandbox" -> null
      - type         = "email" -> null
      - user_labels  = {} -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

I'll have a followup PR to adjust the organization.admin role appropriately to allow this PR to deploy

[spiffxp]

#2010 should hopefully give the right billing permissions to org admins

Does this ensure the ii team doesn't accidentally need billing permissions to manage their project with terraform? Should this be storing state in a different bucket?

[ameukam]

Does this ensure the ii team doesn't accidentally need billing permissions to manage their project with terraform? Should this be storing state in a different bucket?

@spiffxp The budget is supposed to cover only this project. Even if the budget is completely consumed I don't think they will be able to increase the budget limit. Those permissions are at the org. level.

Ultimately, we should move the billing budgets to a dedicated state but for this sandbox I prefer to leave it here. Can move it as a followup if you want.

[ameukam]

I tried again and got the same error : #1975 (comment).

our application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the billingbudgets.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting.

I can't directly use Terraform with my account for this. We could use a service account but I don't think we want this kind of complexity right now.
I'll work on some bash implementation.

[spiffxp]

Uhhh... yeah, I was just browsing the changelog for the terraform google provider

https://github.com/hashicorp/terraform-provider-google/blob/master/CHANGELOG.md#3520-january-11-2021

billing: removed import support for google_billing_budget as it never functioned correctly (#8023)

😅

[spiffxp]

/approve cancel
/lgtm cancel

WDYT about closing this @ameukam ?

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ameukam

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• infra/gcp/clusters/OWNERS [ameukam]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ameukam]

I'll reopen a new PR once we can do terraform apply with postsubmits.
/close

[k8s-ci-robot]

@ameukam: Closed this PR.

Details

In response to this:

I'll reopen a new PR once we can do terraform apply with postsubmits.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.