Skip to content

Conversation

@wking
Copy link
Member

@wking wking commented Oct 1, 2020

We're seeing errors like:

INFO: Unexpected error listing nodes: Get "https://api.ci-op-t47nsmsc-99b10.origin-ci-int-gce.dev.openshift.com:6443/api/v1/nodes?fieldSelector=spec.unschedulable%3Dfalse&resourceVersion=0": dial tcp 35.231.18.254:6443: i/o timeout

in CI jobs running on build02, and are wondering if these are related to the host cluster. Move some promotion informers over to build02 to see if they start experiencing the same symptoms.

Generated with:

$ sed -i 's/cluster: api.ci/cluster: build02/' ci-operator/jobs/openshift/release/openshift-release-release-4.6-periodics.yaml

We're seeing errors like [1]:

  INFO: Unexpected error listing nodes: Get "https://api.ci-op-t47nsmsc-99b10.origin-ci-int-gce.dev.openshift.com:6443/api/v1/nodes?fieldSelector=spec.unschedulable%3Dfalse&resourceVersion=0": dial tcp 35.231.18.254:6443: i/o timeout

in CI jobs running on build02, and are wondering if these are related
to the host cluster.  Move some promotion informers over to build02 to
see if they start experiencing the same symptoms.

Generated with:

  $ sed -i 's/cluster: api.ci/cluster: build02/' ci-operator/jobs/openshift/release/openshift-release-release-4.6-periodics.yaml

[1]: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_ovn-kubernetes/297/pull-ci-openshift-ovn-kubernetes-master-e2e-gcp-ovn/1311628791914696704#1:build-log.txt%3A19
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: wking
To complete the pull request process, please assign bbguimaraes after the PR has been reviewed.
You can assign the PR to them by writing /assign @bbguimaraes in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@alvaroaleman
Copy link
Contributor

You need to update core-services/sanitize-prow-jobs/_config.yaml with the new location. Otherwise lgtm

@wking
Copy link
Member Author

wking commented Oct 1, 2020

@deads2k wasn't excited about moving important promotion jobs onto the suspected cluster. That's fine, and #12386 goes the other way to move jobs off the suspected cluster. Looking ahead, it would be nice if we could shard the release promotion jobs in ci-operator/jobs/openshift/release out over our available clusters so that build-watchers looking at Sippy would see success rates dropping if one of the host clusters had some sort of subtle issue like this. Would that mean dropping this and having per-job entries for each periodic? Or...?

@openshift-ci-robot
Copy link
Contributor

@wking: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/ordered-prow-config 69b8487 link /test ordered-prow-config
ci/prow/boskos-config 69b8487 link /test boskos-config

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@alvaroaleman
Copy link
Contributor

/uncc

@openshift-ci-robot openshift-ci-robot removed the request for review from alvaroaleman October 30, 2020 20:34
@openshift-merge-robot
Copy link
Contributor

@wking: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/release-config 69b8487 link /test release-config
ci/prow/boskos-config-generation 69b8487 link /test boskos-config-generation
ci/prow/secret-generator-config-valid 69b8487 link /test secret-generator-config-valid
ci/prow/deprecate-templates 69b8487 link /test deprecate-templates

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Feb 26, 2021

@wking: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/ci-secret-generator-config 69b8487 link /test ci-secret-generator-config
ci/prow/ci-secret-bootstrap-config-validation 69b8487 link /test ci-secret-bootstrap-config-validation

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-bot
Copy link
Contributor

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@openshift-ci openshift-ci bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 27, 2021
@openshift-bot
Copy link
Contributor

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

@openshift-ci openshift-ci bot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jun 26, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 26, 2021

@wking: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 26, 2021
@openshift-bot
Copy link
Contributor

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

@openshift-ci openshift-ci bot closed this Jul 27, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 27, 2021

@openshift-bot: Closed this PR.

Details

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants