increase gcp-ovn-rt-upgrade timeout from 5h->6h by jluhrsen · Pull Request #69287 · openshift/release

jluhrsen · 2025-09-15T21:08:01Z

it some times will go past 5h and fails, even though its working on post steps and the actual e2e part has passed

martinkennelly · 2025-09-23T16:00:16Z

can you run the job a few times here so we can see the effect of your changes and see if it passes?

jluhrsen · 2025-09-23T16:01:47Z

/pj-reherse pull-ci-openshift-ovn-kubernetes-master-4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade

martinkennelly · 2025-09-23T16:05:51Z

/pj-reherse pull-ci-openshift-ovn-kubernetes-master-4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade

martinkennelly · 2025-09-23T16:25:50Z

doesnt seem to be working jamo

martinkennelly · 2025-09-23T16:42:22Z

/pj-rehearse pull-ci-openshift-ovn-kubernetes-master-4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade

openshift-ci-robot · 2025-09-23T16:42:25Z

@martinkennelly: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

jluhrsen · 2025-09-23T17:08:51Z

doesnt seem to be working jamo

it's working now. pj-rehearse likes to ignore you sometimes. :(

but I think we should
/hold
now that I realize the periodic job is also facing this same issue in the past month. this is something bigger that someone probably needs to look in to. I mean, we can merge this if we want, but I did start a discussion on slack to see if others
know about this and maybe someone is working on it.

martinkennelly · 2025-09-24T12:37:42Z

we can abandon this jamo i think - we saw something in the logs: https://redhat-internal.slack.com/archives/C01CQA76KMX/p1758716716570649?thread_ts=1758647063.563739&cid=C01CQA76KMX

jluhrsen · 2025-09-24T17:14:46Z

we can abandon this jamo i think - we saw something in the logs: https://redhat-internal.slack.com/archives/C01CQA76KMX/p1758716716570649?thread_ts=1758647063.563739&cid=C01CQA76KMX

/close

openshift-ci · 2025-09-24T17:17:46Z

@jluhrsen: Closed this PR.

Details

In response to this:

we can abandon this jamo i think - we saw something in the logs: https://redhat-internal.slack.com/archives/C01CQA76KMX/p1758716716570649?thread_ts=1758647063.563739&cid=C01CQA76KMX

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

martinkennelly · 2025-10-10T11:14:23Z

/reopen

openshift-ci · 2025-10-10T11:14:36Z

@martinkennelly: Reopened this PR.

Details

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

martinkennelly · 2025-10-10T11:15:58Z

@jluhrsen I looked at this job and i think your instincts/analysis were right originally... it does look at the job and it passes the tests after 3 hours but then hits the timeout sometimes. I dont see any other issues.... so it does just look like things are just taking longer and then timing out...

martinkennelly · 2025-10-10T11:16:24Z

/pj-rehearse pull-ci-openshift-ovn-kubernetes-master-4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade

openshift-ci-robot · 2025-10-10T11:16:27Z

@martinkennelly: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

openshift-ci-robot · 2025-10-10T11:21:25Z

@martinkennelly: job(s): pull-ci-openshift-ovn-kubernetes-master-4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade either don't exist or were not found to be affected, and cannot be rehearsed

jluhrsen · 2025-10-15T04:40:34Z

/pj-rehearse pull-ci-openshift-ovn-kubernetes-master-4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade

openshift-ci-robot · 2025-10-15T04:40:36Z

@jluhrsen: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

jluhrsen · 2025-10-15T19:57:26Z

/pj-rehearse pull-ci-openshift-ovn-kubernetes-master-4.21-upgrade-from-stable-4.20-e2e-gcp-ovn-rt-upgrade

openshift-ci-robot · 2025-10-15T19:57:29Z

@jluhrsen: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

martinkennelly · 2025-10-16T12:15:48Z

/lgtm
/approve

TYSM jamo..

martinkennelly · 2025-10-16T12:51:35Z

@jluhrsen can you give your reasoning for including the job openshift-upgrade-aws-single-node-realtime ?

jluhrsen · 2025-10-16T19:23:26Z

single-node-realtime

@martinkennelly , I have fat fingers I guess. unintentional. fixed. need the labels back pls

openshift-ci · 2025-10-16T19:46:15Z

@jluhrsen: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/rehearse/openshift/ovn-kubernetes/master/4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade	`f2517de`	link	unknown	`/pj-rehearse pull-ci-openshift-ovn-kubernetes-master-4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

martinkennelly · 2025-10-16T20:35:07Z

@jluhrsen generated config job isn't happy.

martinkennelly · 2025-10-16T20:35:56Z

/hold

until generated config is good

it some times will go past 5h and fails, even though its working on post steps and the actual e2e part has passed on major factor for these specific jobs is that the build time for OVNK presubmits can take 30-60+ minutes just to build the PR before the job can even start the process of starting a cluster to run the e2e. Signed-off-by: Jamo Luhrsen <jluhrsen@gmail.com>

jluhrsen · 2025-10-16T20:41:44Z

/hold

until generated config is good

@martinkennelly should be good now. forgot to run the make commands after fixing the last fat figure mistake

openshift-ci-robot · 2025-10-16T20:43:46Z

[REHEARSALNOTIFIER]
@jluhrsen: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
pull-ci-openshift-ovn-kubernetes-master-4.21-upgrade-from-stable-4.20-e2e-gcp-ovn-rt-upgrade	openshift/ovn-kubernetes	presubmit	Ci-operator config changed
pull-ci-openshift-ovn-kubernetes-release-4.18-4.18-upgrade-from-stable-4.17-e2e-gcp-ovn-rt-upgrade	openshift/ovn-kubernetes	presubmit	Ci-operator config changed
pull-ci-openshift-ovn-kubernetes-release-4.20-4.20-upgrade-from-stable-4.19-e2e-gcp-ovn-rt-upgrade	openshift/ovn-kubernetes	presubmit	Ci-operator config changed
pull-ci-openshift-ovn-kubernetes-release-4.19-4.19-upgrade-from-stable-4.18-e2e-gcp-ovn-rt-upgrade	openshift/ovn-kubernetes	presubmit	Ci-operator config changed

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

martinkennelly · 2025-10-16T21:03:21Z

/lgtm

openshift-ci · 2025-10-16T21:03:46Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jluhrsen, martinkennelly

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~ci-operator/config/openshift/ovn-kubernetes/OWNERS~~ [martinkennelly]
~~ci-operator/jobs/openshift/ovn-kubernetes/OWNERS~~ [martinkennelly]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

martinkennelly · 2025-10-16T21:07:53Z

/rehearsals-ack

martinkennelly · 2025-10-16T21:17:51Z

/rehearse ack

martinkennelly · 2025-10-16T21:30:24Z

@jluhrsen can you ack rehearsals? We've already verified it.

martinkennelly · 2025-10-16T21:49:21Z

/pj-rehearse ack

openshift-ci-robot · 2025-10-16T21:49:23Z

@martinkennelly: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

martinkennelly · 2025-10-16T22:09:29Z

/unhold

jluhrsen mentioned this pull request Sep 15, 2025

OCPBUGS-58501,OCPBUGS-59657,OCPBUGS-42303,OCPBUGS-61566: DownStream Merge [09-09-2025] openshift/ovn-kubernetes#2750

Merged

openshift-ci bot requested review from jcaamano and kyrtapz September 15, 2025 21:10

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 23, 2025

openshift-ci bot closed this Sep 24, 2025

martinkennelly mentioned this pull request Oct 10, 2025

OCPBUGS-62545: Updating ose-ovn-kubernetes-container image to be consistent with ART for 4.21 openshift/ovn-kubernetes#2771

Merged

openshift-ci bot reopened this Oct 10, 2025

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 10, 2025

This was referenced Oct 10, 2025

OCPBUGS-60781, CORENET-6055: Dockerfile: Unpin OVN and consume the latest from FDP openshift/ovn-kubernetes#2721

Merged

CORENET-6160: DownStream Merge [10-10-2025] openshift/ovn-kubernetes#2798

Merged

jluhrsen force-pushed the increase-gcp-ovn-rt-upgrade-timeout branch from f2517de to 613f069 Compare October 15, 2025 04:36

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 15, 2025

openshift-ci bot assigned martinkennelly Oct 16, 2025

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Oct 16, 2025

jcaamano mentioned this pull request Oct 16, 2025

OCPBUGS-62913: Configure sec nic EIPv6 address with NODAD and maximum lifetime openshift/ovn-kubernetes#2797

Merged

jluhrsen force-pushed the increase-gcp-ovn-rt-upgrade-timeout branch from 44708b8 to 70b5881 Compare October 16, 2025 19:22

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Oct 16, 2025

jluhrsen force-pushed the increase-gcp-ovn-rt-upgrade-timeout branch from 70b5881 to 6b120d9 Compare October 16, 2025 20:41

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 16, 2025

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Oct 16, 2025

openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Oct 16, 2025

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 16, 2025

openshift-merge-bot bot merged commit e63e209 into openshift:master Oct 16, 2025
15 checks passed

Comments

Conversation

jluhrsen commented Sep 15, 2025

Uh oh!

martinkennelly commented Sep 23, 2025

Uh oh!

jluhrsen commented Sep 23, 2025

Uh oh!

martinkennelly commented Sep 23, 2025

Uh oh!

martinkennelly commented Sep 23, 2025

Uh oh!

martinkennelly commented Sep 23, 2025

Uh oh!

openshift-ci-robot commented Sep 23, 2025

Uh oh!

jluhrsen commented Sep 23, 2025

Uh oh!

martinkennelly commented Sep 24, 2025

Uh oh!

jluhrsen commented Sep 24, 2025

Uh oh!

openshift-ci bot commented Sep 24, 2025

Uh oh!

martinkennelly commented Oct 10, 2025

Uh oh!

openshift-ci bot commented Oct 10, 2025

Uh oh!

martinkennelly commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

martinkennelly commented Oct 10, 2025

Uh oh!

openshift-ci-robot commented Oct 10, 2025

Uh oh!

openshift-ci-robot commented Oct 10, 2025

Uh oh!

jluhrsen commented Oct 15, 2025

Uh oh!

openshift-ci-robot commented Oct 15, 2025

Uh oh!

jluhrsen commented Oct 15, 2025

Uh oh!

openshift-ci-robot commented Oct 15, 2025

Uh oh!

martinkennelly commented Oct 16, 2025

Uh oh!

martinkennelly commented Oct 16, 2025

Uh oh!

jluhrsen commented Oct 16, 2025

Uh oh!

openshift-ci bot commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

martinkennelly commented Oct 16, 2025

Uh oh!

martinkennelly commented Oct 16, 2025

Uh oh!

jluhrsen commented Oct 16, 2025

Uh oh!

openshift-ci-robot commented Oct 16, 2025

Uh oh!

martinkennelly commented Oct 16, 2025

Uh oh!

openshift-ci bot commented Oct 16, 2025

Uh oh!

martinkennelly commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

martinkennelly commented Oct 16, 2025

Uh oh!

martinkennelly commented Oct 16, 2025

Uh oh!

martinkennelly commented Oct 16, 2025

Uh oh!

openshift-ci-robot commented Oct 16, 2025

Uh oh!

martinkennelly commented Oct 16, 2025

Uh oh!

Uh oh!

martinkennelly commented Oct 10, 2025 •

edited

Loading

openshift-ci bot commented Oct 16, 2025 •

edited

Loading

martinkennelly commented Oct 16, 2025 •

edited

Loading