Added filtering for incorrect slo boosting logs by chrisThePattyEater · Pull Request #2118 · kubernetes-sigs/gcp-compute-persistent-disk-csi-driver

chrisThePattyEater · 2025-06-27T02:38:38Z

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change
/kind bug

/kind cleanup

/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

What this PR does / why we need it:
Recently we were out of SLO for the PDCSI CreateSnapshot endpoint. After analysis of the errors contributing to our slo we have identified the issue to be an excessive amount of INTERNAL errors. Further analysis of these internal errors has brought to light two major types of errors that are being incorrectly tagged as internal. These should not be contributing to our slos, they are the following:
Already exists errors should be converted from INTERNAL to ALREADY_EXISTS
Org policy violations should be converted from INTERNAL to FAILED_PRECONDITION

These errors are both returned from the create snapshot gcp endpoint

This change introduces methods that use regex to identify these two types of errors in the response payload and correctly marks them as ALREADY_EXISTS and FAILED_PRECONDITION errors.

Which issue(s) this PR fixes:

Fixes #
https://b.corp.google.com/issues/398008774
Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

linux-foundation-easycla · 2025-06-27T02:38:42Z

The committers listed above are authorized under a signed CLA.

✅ login: dannawang0221 / name: dannawang0221 (0fb50d2)
✅ login: chrisThePattyEater / name: Chris Fuentes (f015679)

k8s-ci-robot · 2025-06-27T02:38:47Z

Welcome @chrisThePattyEater!

It looks like this is your first PR to kubernetes-sigs/gcp-compute-persistent-disk-csi-driver 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/gcp-compute-persistent-disk-csi-driver has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2025-06-27T02:38:48Z

Hi @chrisThePattyEater. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

pwschuurman · 2025-06-30T18:49:11Z

+	if err == nil {
+		return false
+	}
+	re, compileErr := regexp.Compile(regexPattern)


Can you set re as a static var, with regexp.MustCompile?

pwschuurman · 2025-06-30T19:00:42Z

+
+// IsGCPOrgViolationError checks if the error is a GCP organization policy violation error.
+func IsGCPOrgViolationError(err error) bool {
+	return ErrorContainsRegex(err, "violates constraint constraints/gcp.")


Is there a more complex error policy violation field in the response that you can check instead of string error message extraction?

The risk is if the error message from the API is reformatted, this regex matching may fail. If there is a more dependable API type response we could use, I would prefer to use that.

pwschuurman · 2025-06-30T19:01:20Z

+		expectedResult bool
+	}{
+		{
+			name:           "is gcp org violation error",


Test name does not match Test function.

dannawang0221 · 2025-08-11T17:19:22Z

/ok-to-test

dannawang0221 · 2025-08-14T08:31:22Z

/retest

k8s-ci-robot · 2025-08-14T08:34:56Z

@chrisThePattyEater: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-gcp-compute-persistent-disk-csi-driver-e2e-arm	`0fb50d2`	link	false	`/test pull-gcp-compute-persistent-disk-csi-driver-e2e-arm`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

k8s-ci-robot · 2025-08-14T22:53:23Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cemakd, chrisThePattyEater, pwschuurman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [pwschuurman]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pwschuurman · 2025-08-14T22:53:27Z

/lgtm

cemakd · 2025-09-25T17:09:11Z

/cherry-pick release-1.21

k8s-infra-cherrypick-robot · 2025-09-25T17:09:48Z

@cemakd: new pull request created: #2181

Details

In response to this:

/cherry-pick release-1.21

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. labels Jun 27, 2025

k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Jun 27, 2025

k8s-ci-robot requested review from pwschuurman and saikat-royc June 27, 2025 02:38

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jun 27, 2025

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Jun 27, 2025

cemakd reviewed Jun 27, 2025

View reviewed changes

Comment thread pkg/gce-cloud-provider/compute/gce.go Outdated

cemakd reviewed Jun 27, 2025

View reviewed changes

Comment thread pkg/gce-pd-csi-driver/controller.go Outdated

chrisThePattyEater force-pushed the master branch 2 times, most recently from 5c1e6e8 to e42e6e2 Compare June 27, 2025 18:41

chrisThePattyEater requested a review from cemakd June 27, 2025 18:46

cemakd approved these changes Jun 30, 2025

View reviewed changes

pwschuurman reviewed Jun 30, 2025

View reviewed changes

Added filtering for incorrect slo boosting logs

f015679

chrisThePattyEater force-pushed the master branch from e42e6e2 to f015679 Compare July 2, 2025 17:55

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 2, 2025

Update org policy violation check

0fb50d2

pwschuurman approved these changes Jul 17, 2025

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 17, 2025

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 11, 2025

pwschuurman approved these changes Aug 14, 2025

View reviewed changes

k8s-ci-robot assigned pwschuurman Aug 14, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 14, 2025

pwschuurman merged commit 127d813 into kubernetes-sigs:master Aug 14, 2025
8 of 10 checks passed

k8s-infra-cherrypick-robot mentioned this pull request Sep 25, 2025

[release-1.21] Added filtering for incorrect slo boosting logs #2181

Merged

Conversation

chrisThePattyEater commented Jun 27, 2025

Uh oh!

linux-foundation-easycla Bot commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Jun 27, 2025

Uh oh!

k8s-ci-robot commented Jun 27, 2025

Uh oh!

Uh oh!

Uh oh!

pwschuurman Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

pwschuurman Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

pwschuurman Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

dannawang0221 commented Aug 11, 2025

Uh oh!

dannawang0221 commented Aug 14, 2025

Uh oh!

k8s-ci-robot commented Aug 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Aug 14, 2025

Uh oh!

pwschuurman commented Aug 14, 2025

Uh oh!

Uh oh!

cemakd commented Sep 25, 2025

Uh oh!

k8s-infra-cherrypick-robot commented Sep 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

linux-foundation-easycla Bot commented Jun 27, 2025 •

edited

Loading

k8s-ci-robot commented Aug 14, 2025 •

edited

Loading