GCP-441: correct CCM crash toleration key to match pod label

[cristianoveiga]

What this PR does / why we need it:

Fixes the CCM crash toleration key in the e2e test suite. The podCrashTolerations map used "gcp-cloud-controller-manager" as the key, but getComponentName() returns the pod's app label which is "cloud-controller-manager" for all CCM providers (AWS, Azure, GCP, KubeVirt). This caused the toleration added in GCP-441 to never match, leaving the GCP CCM token-minter race condition unmitigated in CI.

Which issue(s) this PR fixes:

Fixes https://issues.redhat.com/browse/GCP-441

Special notes for your reviewer:

This is a temporary workaround until https://issues.redhat.com/browse/GCP-447 is implemented, which will inject the token-minter as a native sidecar init container with a StartupProbe, eliminating the race condition entirely.

A TODO(GCP-447) comment has been added to track the removal of this toleration.

Note that the fix applies the toleration to all CCM providers (AWS, Azure, GCP, KubeVirt) since they all share the same app: cloud-controller-manager label. This is acceptable because the token-minter race condition affects all providers — it has only been observed in GCP CI due to the GCP CCM's fail-fast initialization.

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Summary by CodeRabbit

• Chores
  • Updated internal testing infrastructure to use standardized Cloud Controller Manager configuration across providers.
  • Renamed the deployment's app label for the GCP Cloud Controller Manager to align ownership/selection with the standardized naming.
  • Added a non-functional TODO note in test utilities (no behavior changes).

[openshift-ci-robot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

[openshift-ci-robot]

@cristianoveiga: This pull request references GCP-441 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Fixes the CCM crash toleration key in the e2e test suite. The podCrashTolerations map used "gcp-cloud-controller-manager" as the key, but getComponentName() returns the pod's app label which is "cloud-controller-manager" for all CCM providers (AWS, Azure, GCP, KubeVirt). This caused the toleration added in GCP-441 to never match, leaving the GCP CCM token-minter race condition unmitigated in CI.

Which issue(s) this PR fixes:

Fixes https://issues.redhat.com/browse/GCP-441

Special notes for your reviewer:

This is a temporary workaround until https://issues.redhat.com/browse/GCP-447 is implemented, which will inject the token-minter as a native sidecar init container with a StartupProbe, eliminating the race condition entirely.

A TODO(GCP-447) comment has been added to track the removal of this toleration.

Note that the fix applies the toleration to all CCM providers (AWS, Azure, GCP, KubeVirt) since they all share the same app: cloud-controller-manager label. This is acceptable because the token-minter race condition affects all providers — it has only been observed in GCP CI due to the GCP CCM's fail-fast initialization.

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[openshift-ci-robot]

@cristianoveiga: This pull request references GCP-441 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Fixes the CCM crash toleration key in the e2e test suite. The podCrashTolerations map used "gcp-cloud-controller-manager" as the key, but getComponentName() returns the pod's app label which is "cloud-controller-manager" for all CCM providers (AWS, Azure, GCP, KubeVirt). This caused the toleration added in GCP-441 to never match, leaving the GCP CCM token-minter race condition unmitigated in CI.

Which issue(s) this PR fixes:

Fixes https://issues.redhat.com/browse/GCP-441

Special notes for your reviewer:

This is a temporary workaround until https://issues.redhat.com/browse/GCP-447 is implemented, which will inject the token-minter as a native sidecar init container with a StartupProbe, eliminating the race condition entirely.

A TODO(GCP-447) comment has been added to track the removal of this toleration.

Note that the fix applies the toleration to all CCM providers (AWS, Azure, GCP, KubeVirt) since they all share the same app: cloud-controller-manager label. This is acceptable because the token-minter race condition affects all providers — it has only been observed in GCP CI due to the GCP CCM's fail-fast initialization.

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

📝 Walkthrough

Walkthrough

Added a TODO comment in the test utility (test/e2e/util/util.go) describing removal of a toleration entry from the podCrashTolerations map. Updated the GCP cloud-controller-manager Deployment manifest (control-plane-operator/.../deployment.yaml) to change the app label used for selector and template from app: cloud-controller-manager to app: gcp-cloud-controller-manager. No other functional or API-surface changes were introduced.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: correcting the CCM crash toleration key to match pod labels.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Stable And Deterministic Test Names	✅ Passed	This PR does not modify any Ginkgo test names. Changes consist of a TODO comment in test utility code and label selectors in a Kubernetes deployment manifest.
Test Structure And Quality	✅ Passed	PR does not modify Ginkgo test code; changes only affect test utility configuration and Kubernetes manifests.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[openshift-ci-robot]

@cristianoveiga: This pull request references GCP-441 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Fixes the CCM crash toleration key in the e2e test suite. The podCrashTolerations map used "gcp-cloud-controller-manager" as the key, but getComponentName() returns the pod's app label which is "cloud-controller-manager" for all CCM providers (AWS, Azure, GCP, KubeVirt). This caused the toleration added in GCP-441 to never match, leaving the GCP CCM token-minter race condition unmitigated in CI.

Which issue(s) this PR fixes:

Fixes https://issues.redhat.com/browse/GCP-441

Special notes for your reviewer:

This is a temporary workaround until https://issues.redhat.com/browse/GCP-447 is implemented, which will inject the token-minter as a native sidecar init container with a StartupProbe, eliminating the race condition entirely.

A TODO(GCP-447) comment has been added to track the removal of this toleration.

Note that the fix applies the toleration to all CCM providers (AWS, Azure, GCP, KubeVirt) since they all share the same app: cloud-controller-manager label. This is acceptable because the token-minter race condition affects all providers — it has only been observed in GCP CI due to the GCP CCM's fail-fast initialization.

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@cristianoveiga: This pull request references GCP-441 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Fixes the CCM crash toleration key in the e2e test suite. The podCrashTolerations map used "gcp-cloud-controller-manager" as the key, but getComponentName() returns the pod's app label which is "cloud-controller-manager" for all CCM providers (AWS, Azure, GCP, KubeVirt). This caused the toleration added in GCP-441 to never match, leaving the GCP CCM token-minter race condition unmitigated in CI.

Which issue(s) this PR fixes:

Fixes https://issues.redhat.com/browse/GCP-441

Special notes for your reviewer:

This is a temporary workaround until https://issues.redhat.com/browse/GCP-447 is implemented, which will inject the token-minter as a native sidecar init container with a StartupProbe, eliminating the race condition entirely.

A TODO(GCP-447) comment has been added to track the removal of this toleration.

Note that the fix applies the toleration to all CCM providers (AWS, Azure, GCP, KubeVirt) since they all share the same app: cloud-controller-manager label. This is acceptable because the token-minter race condition affects all providers — it has only been observed in GCP CI due to the GCP CCM's fail-fast initialization.

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@cristianoveiga: This pull request references GCP-441 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Fixes the CCM crash toleration key in the e2e test suite. The podCrashTolerations map used "gcp-cloud-controller-manager" as the key, but getComponentName() returns the pod's app label which is "cloud-controller-manager" for all CCM providers (AWS, Azure, GCP, KubeVirt). This caused the toleration added in GCP-441 to never match, leaving the GCP CCM token-minter race condition unmitigated in CI.

Which issue(s) this PR fixes:

Fixes https://issues.redhat.com/browse/GCP-441

Special notes for your reviewer:

This is a temporary workaround until https://issues.redhat.com/browse/GCP-447 is implemented, which will inject the token-minter as a native sidecar init container with a StartupProbe, eliminating the race condition entirely.

A TODO(GCP-447) comment has been added to track the removal of this toleration.

Note that the fix applies the toleration to all CCM providers (AWS, Azure, GCP, KubeVirt) since they all share the same app: cloud-controller-manager label. This is acceptable because the token-minter race condition affects all providers — it has only been observed in GCP CI due to the GCP CCM's fail-fast initialization.

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Summary by CodeRabbit

• Chores
• Updated internal testing infrastructure to use standardized Cloud Controller Manager configuration across providers.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@test/e2e/util/util.go`:
- Around line 137-139: EnsureNoCrashingPods wrongly collapses provider-specific
CCM tolerations because getComponentName() prefers pod.Labels["app"], so the
"kubevirt-cloud-controller-manager": 2 entry can never match the new
"cloud-controller-manager" app label; fix by making getComponentName() prefer
provider-aware labels (check in order e.g. pod.Labels["app.kubernetes.io/name"],
pod.Labels["kubevirt.io/component"] or any provider-specific label) before
falling back to pod.Labels["app"], and/or update the podCrashTolerations map to
include both the generic "cloud-controller-manager" and the provider-specific
key (kubevirt-cloud-controller-manager) so the stricter 2-restart allowance for
KubeVirt is honored by EnsureNoCrashingPods.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro

Run ID: a1a53851-9fd9-4104-a526-55cd7a78c499

📥 Commits

Reviewing files that changed from the base of the PR and between 5271846 and a581e40.

📒 Files selected for processing (1)

• test/e2e/util/util.go

[openshift-ci-robot]

@cristianoveiga: This pull request references GCP-441 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Fixes the CCM crash toleration key in the e2e test suite. The podCrashTolerations map used "gcp-cloud-controller-manager" as the key, but getComponentName() returns the pod's app label which is "cloud-controller-manager" for all CCM providers (AWS, Azure, GCP, KubeVirt). This caused the toleration added in GCP-441 to never match, leaving the GCP CCM token-minter race condition unmitigated in CI.

Which issue(s) this PR fixes:

Fixes https://issues.redhat.com/browse/GCP-441

Special notes for your reviewer:

This is a temporary workaround until https://issues.redhat.com/browse/GCP-447 is implemented, which will inject the token-minter as a native sidecar init container with a StartupProbe, eliminating the race condition entirely.

A TODO(GCP-447) comment has been added to track the removal of this toleration.

Note that the fix applies the toleration to all CCM providers (AWS, Azure, GCP, KubeVirt) since they all share the same app: cloud-controller-manager label. This is acceptable because the token-minter race condition affects all providers — it has only been observed in GCP CI due to the GCP CCM's fail-fast initialization.

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Summary by CodeRabbit

• Chores
• Updated internal testing infrastructure to use standardized Cloud Controller Manager configuration across providers.
• Renamed the deployment's app label for the GCP Cloud Controller Manager to align ownership/selection with the standardized naming.
• Added a non-functional TODO note in test utilities (no behavior changes).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@cristianoveiga: This pull request references GCP-441 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Fixes the CCM crash toleration key in the e2e test suite. The podCrashTolerations map used "gcp-cloud-controller-manager" as the key, but getComponentName() returns the pod's app label which is "cloud-controller-manager" for all CCM providers (AWS, Azure, GCP, KubeVirt). This caused the toleration added in GCP-441 to never match, leaving the GCP CCM token-minter race condition unmitigated in CI.

Which issue(s) this PR fixes:

Fixes https://issues.redhat.com/browse/GCP-441

Special notes for your reviewer:

This is a temporary workaround until https://issues.redhat.com/browse/GCP-447 is implemented, which will inject the token-minter as a native sidecar init container with a StartupProbe, eliminating the race condition entirely.

A TODO(GCP-447) comment has been added to track the removal of this toleration.

Note that the fix applies the toleration to all CCM providers (AWS, Azure, GCP, KubeVirt) since they all share the same app: cloud-controller-manager label. This is acceptable because the token-minter race condition affects all providers — it has only been observed in GCP CI due to the GCP CCM's fail-fast initialization.

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Summary by CodeRabbit

• Chores
• Updated internal testing infrastructure to use standardized Cloud Controller Manager configuration across providers.
• Renamed the deployment's app label for the GCP Cloud Controller Manager to align ownership/selection with the standardized naming.
• Added a non-functional TODO note in test utilities (no behavior changes).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[cblecker]

/lgtm

[openshift-ci-robot]

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-21
/test e2e-aws-4-21
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws

[cristianoveiga]

/verified by @cristianoveiga

[openshift-ci-robot]

@cristianoveiga: This PR has been marked as verified by @cristianoveiga.

Details

In response to this:

/verified by @cristianoveiga

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[cwbotbot]

Test Results

e2e-aws

• Status: ❌ FAIL
• Started: 2026-03-12T13:55:12Z
• View Job
• View Job History

Failed Tests

Total failed tests: 24

• TestAutoscaling
• TestAutoscaling/ValidateHostedCluster
• TestCreateCluster
• TestCreateCluster/ValidateHostedCluster
• TestCreateClusterCustomConfig

... and 19 more failed tests

e2e-aks

• Status: ✅ PASS
• Started: 2026-03-13T19:48:04Z
• View Job
• View Job History

[cblecker]

/retest

[cristianoveiga]

/retest

[sjenning]

/approve

[sjenning]

/retest-required

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cristianoveiga, sjenning

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [sjenning]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@cristianoveiga: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.