Skip to content

bug 2005901: Allow KA guard probe to fail as designed#26766

Merged
deads2k merged 1 commit intoopenshift:masterfrom
ingvagabund:synthetic-allow-ka-guard-pod-probe-ready-fail
Jan 19, 2022
Merged

bug 2005901: Allow KA guard probe to fail as designed#26766
deads2k merged 1 commit intoopenshift:masterfrom
ingvagabund:synthetic-allow-ka-guard-pod-probe-ready-fail

Conversation

@ingvagabund
Copy link
Member

To address

event happened 47 times, something is wrong: ns/openshift-kube-apiserver pod/kube-apiserver-guard-ci-op-h9q0w0xg-14a78-7gf8z-master-0 node/ci-op-h9q0w0xg-14a78-7gf8z-master-0 - reason/ProbeError Readiness probe error: Get "https://10.0.0.8:6443/healthz": dial tcp 10.0.0.8:6443: connect: connection refused
body:
event happened 47 times, something is wrong: ns/openshift-kube-apiserver pod/kube-apiserver-guard-ci-op-h9q0w0xg-14a78-7gf8z-master-0 node/ci-op-h9q0w0xg-14a78-7gf8z-master-0 - reason/Unhealthy Readiness probe failed: Get "https://10.0.0.8:6443/healthz": dial tcp 10.0.0.8:6443: connect: connection refused

E.g. https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.10-e2e-azure-ovn-upgrade/1482186672689909760

The KA guard pod probe is expected to fail as its readiness depends on KA operand to be ready. Which may not always hold during the bootstrapping phase.

@ingvagabund ingvagabund changed the title Allow KA guard probe to fail as designed bug 2005901: Allow KA guard probe to fail as designed Jan 19, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 19, 2022

@ingvagabund: An error was encountered querying GitHub for users with public email (knarra@redhat.com) for bug 2005901 on the Bugzilla server at https://bugzilla.redhat.com. No known errors were detected, please see the full error message for details.

Full error message. non-200 OK status code: 403 Forbidden body: "{\n \"documentation_url\": \"https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits\",\n \"message\": \"You have exceeded a secondary rate limit. Please wait a few minutes before you try again.\"\n}\n"

Please contact an administrator to resolve this issue, then request a bug refresh with /bugzilla refresh.

Details

In response to this:

bug 2005901: Allow KA guard probe to fail as designed

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

regexp.MustCompile("ns/loki pod/loki-promtail.*Readiness probe failed"),

// kube-apiserver guard probe failing due to kube-apiserver operands getting rolled out
// multiple times during the bootstrapping phase of a cluster installation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and notably, it's the same pod name each time

@deads2k
Copy link
Contributor

deads2k commented Jan 19, 2022

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 19, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 19, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, ingvagabund

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 19, 2022
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

2 similar comments
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@ingvagabund
Copy link
Member Author

/bugzilla refresh

@openshift-ci openshift-ci bot added the bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. label Jan 19, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 19, 2022

@ingvagabund: This pull request references Bugzilla bug 2005901, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.10.0) matches configured target release for branch (4.10.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Requesting review from QA contact:
/cc @kasturinarra

Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Jan 19, 2022
@openshift-ci openshift-ci bot requested a review from kasturinarra January 19, 2022 20:10
@deads2k
Copy link
Contributor

deads2k commented Jan 19, 2022

tested green before. doesn't conflict. cleaning up our signal

@deads2k deads2k merged commit 3c65dcc into openshift:master Jan 19, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 19, 2022

@ingvagabund: Some pull requests linked via external trackers have merged:

The following pull requests linked via external trackers have not merged:

These pull request must merge or be unlinked from the Bugzilla bug in order for it to move to the next state. Once unlinked, request a bug refresh with /bugzilla refresh.

Bugzilla bug 2005901 has not been moved to the MODIFIED state.

Details

In response to this:

bug 2005901: Allow KA guard probe to fail as designed

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 19, 2022

@ingvagabund: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-single-node d0bf342 link false /test e2e-aws-single-node

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ingvagabund ingvagabund deleted the synthetic-allow-ka-guard-pod-probe-ready-fail branch January 19, 2022 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments