Skip to content

Add project for gcs audits logs#2031

Merged
k8s-ci-robot merged 2 commits intokubernetes:mainfrom
ameukam:audit-pii-project-tf
Jun 14, 2021
Merged

Add project for gcs audits logs#2031
k8s-ci-robot merged 2 commits intokubernetes:mainfrom
ameukam:audit-pii-project-tf

Conversation

@ameukam
Copy link
Copy Markdown
Member

@ameukam ameukam commented May 10, 2021 •
edited
Loading

  • Dedicated GCP project for PII
  • BigQuery dataset for PII analysis
  • Bucket for GCR container registries audit logs

Signed-off-by: Arnaud Meukam ameukam@gmail.com

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 10, 2021
@k8s-ci-robot k8s-ci-robot requested review from dims and spiffxp May 10, 2021 23:13
@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. area/terraform Terraform modules, testing them, writing more of them, code in infra/gcp/clusters/ wg/k8s-infra size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 10, 2021
This was referenced May 13, 2021
Closed
Closed
Merged
spiffxp
spiffxp suggested changes May 19, 2021
View reviewed changes
infra/gcp/clusters/projects/k8s-infra-public-pii/main.tf Outdated
}

// Create a sink for the organization
resource "google_logging_organization_sink" "gcs-logs-org-sink" {
Copy link
Copy Markdown
Contributor

@spiffxp spiffxp May 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not familiar with this so no opinion

Copy link
Copy Markdown
Member Author

@ameukam ameukam May 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking at Aggregated sinks but I realized I don't need this.

@ameukam ameukam force-pushed the audit-pii-project-tf branch from 4a1da72 to 022fbb1 Compare May 20, 2021 18:36
ameukam
ameukam commented May 20, 2021
View reviewed changes
ameukam
ameukam commented May 20, 2021
View reviewed changes
spiffxp
spiffxp suggested changes Jun 8, 2021
View reviewed changes
infra/gcp/clusters/projects/k8s-infra-public-pii/provider.tf Outdated
terraform {

backend "gcs" {
bucket = "k8s-infra-tf-public-ii"
Copy link
Copy Markdown
Contributor

@spiffxp spiffxp Jun 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe I'm misunderstanding the intent here. I assumed this was about PII, not about the ii folks having a dedicated project in public

Suggested change
bucket = "k8s-infra-tf-public-ii"
bucket = "k8s-infra-tf-public-pii"

Copy link
Copy Markdown
Member Author

@ameukam ameukam Jun 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

ameukam added 2 commits June 11, 2021 00:39
@ameukam
Add project for gcs audits logs
ebd46a9 
- Dedicated GCP project for PII
- BigQuery dataset for PII analysis
- Bucket for GCR container registries audit logs

Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
@ameukam
Ensure gcs bucket for Terraform state of k8s-infra-public-ii
344e2dc 
Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
@ameukam ameukam force-pushed the audit-pii-project-tf branch from 022fbb1 to 344e2dc Compare June 10, 2021 22:50
@ameukam ameukam changed the title [WIP] Add project for gcs audits logs Add project for gcs audits logs Jun 10, 2021
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 10, 2021
@spiffxp spiffxp mentioned this pull request Jun 11, 2021
Closed
spiffxp
spiffxp approved these changes Jun 14, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@spiffxp spiffxp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 14, 2021
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

k8s-ci-robot commented Jun 14, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ameukam, spiffxp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 3877257 into kubernetes:main Jun 14, 2021
@k8s-ci-robot k8s-ci-robot added this to the v1.22 milestone Jun 14, 2021
@ameukam
Copy link
Copy Markdown
Member Author

ameukam commented Jun 14, 2021

Running ./ensure-main-project.sh

@spiffxp
Copy link
Copy Markdown
Contributor

spiffxp commented Jun 14, 2021

Ensuring 'gs://k8s-infra-tf-public-pii' exists as private with owners 'k8s-infra-cluster-admins@kubernetes.io'

Didn't catch this during review... this is too broad a group. I will open a followup to move this to org-admins once this has been deployed

@spiffxp
Copy link
Copy Markdown
Contributor

spiffxp commented Jun 14, 2021

@ameukam I was going to run terraform apply but I'm going to hold off if you're taking on deployment

@spiffxp spiffxp mentioned this pull request Jun 14, 2021
Merged
@ameukam
Copy link
Copy Markdown
Member Author

ameukam commented Jun 14, 2021

Got during the deployment :

Error: Error setting IAM policy for storage bucket "b/k8s-infra-artifacts-gcslogs": googleapi: Error 409: The metadata for object "null" was edited during the operation. Please try again., conflict

  on main.tf line 114, in resource "google_storage_bucket_iam_policy" "analytics_legacybucketwriter_policy":
 114: resource "google_storage_bucket_iam_policy" "analytics_legacybucketwriter_policy" {

I think the bindings with cloud-storage-analytics@google.com as member are in conflict.

@ameukam ameukam mentioned this pull request Jun 14, 2021
Merged
@bernokl bernokl mentioned this pull request Jun 16, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dims dims Awaiting requested review from dims

1 more reviewer

@spiffxp spiffxp spiffxp approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@spiffxp spiffxp

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/terraform Terraform modules, testing them, writing more of them, code in infra/gcp/clusters/ cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

v1.22

Development

Successfully merging this pull request may close these issues.

3 participants

@ameukam @k8s-ci-robot @spiffxp