infra/gcp: enable GCS access logs for staging project GCS and GCR

[bernokl]

Addresses:
#904 (comment)
Intent:

• Ensure a bucket exist for the logs inside each project
• Enable logging for k8s-staging-* buckets
• Ensure access is gated to limit access per Add project for gcs audits logs #2031
  Research:
  https://github.com/ii/org/blob/main/research/k8sio-ensure-staging-pii-gcs.org

[k8s-ci-robot]

Hi @bernokl. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[spiffxp]

Thanks for starting on this! I added suggestions on how to implement

[spiffxp]

Getting there, more suggestions

[spiffxp]

/ok-to-test

[spiffxp]

This should hopefully be the last round

[spiffxp]

For parity with other functions in this library, give an example, and stick with bucket vs. bucket name

Suggested change

	# $1: The GCS bucket
	function ensure_gcs_bucket_logging() {
	if [ $# != 2 ] || [ -z "$1" ]; then
	echo "ensure_gcs_bucket_logging(name) requires 1 argument" >&2
	return 1
	fi
	local bucket="$1"
	# $1: The GCS bucket (e.g. gs://k8s-infra-foo)
	function ensure_gcs_bucket_logging() {
	if [ $# != 2 ] || [ -z "$1" ]; then
	echo "ensure_gcs_bucket_logging(bucket) requires 1 argument" >&2
	return 1
	fi
	local bucket="$1"

[spiffxp]

whoops, one more... it should be [ $# != 1 ]

[spiffxp]

Please address shellcheck failures that are causing the verify job to fail

[spiffxp]

Since we're getting close, I would also prefer to see this rebased to get rid of the fixup commits, and into commits relevant to the appropriate files

• infra/gcp/lib: add gcs logs bucket
• infra/gcp/lib_gcs: add ensure_gcs_bucket_logging
• infra/gcp/staging: enable gcs logs for staging buckets

[spiffxp]

/approve
/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bernokl, spiffxp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [spiffxp]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[spiffxp]

Running into a few issues while trying to deploy these changes, opened #2283 with the necessary fixes to deploy

[spiffxp]

Running ./infra/gcp/ensure-staging-storage.sh to deploy changes for all staging projects

[spiffxp]

Finished deploying