Create CNV scenario to run `localnet` test on Goldman Sachs configured bare-metal cluster. #71414

dfrazzette · 2025-11-16T07:05:30Z

Create OpenShift Virtualization scenario to run localnet test on bare-metal cluster configured for Goldman Sachs biweekly (currently available to Interop every other week; ref . OCP Goldman Sachs Bare-Metal Cluster Schedule.

dfrazzette · 2025-11-16T07:06:08Z

/hold

dfrazzette · 2025-11-16T07:10:40Z

/pj-rehearse periodic-ci-RedHatQE-interop-testing-master-gs-baremetal-localnet-ocp4.19-lp-gs-gs-baremetal-localnet-ocp419

openshift-ci-robot · 2025-11-16T07:10:42Z

@dfrazzette: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

dfrazzette · 2025-11-16T19:47:30Z

/pj-rehearse periodic-ci-RedHatQE-interop-testing-master-gs-baremetal-localnet-ocp4.19-lp-gs-gs-baremetal-localnet-ocp419

openshift-ci-robot · 2025-11-16T19:47:32Z

@dfrazzette: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

dfrazzette · 2025-11-18T03:44:49Z

/unhold

dfrazzette · 2025-11-18T04:06:54Z

The test is designed for OCP 4.19 or above. The current state of GS BM Cluster is only at 4.16; the customer GS themselves are still at 4.18 with upgrade to 4.19 plan in their schedule.

The last rehearsal job run failed with:

+ oc patch storageclass ocs-storagecluster-ceph-rbd-virtualization -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "true"}}}'
Error from server (NotFound): storageclasses.storage.k8s.io "ocs-storagecluster-ceph-rbd-virtualization" not found

However, Interop no longer has access to the bare-metal cluster until Monday, so will debug this error in scheduled test run once merged.

/pj-rehearse ack

openshift-ci-robot · 2025-11-18T04:06:57Z

@dfrazzette: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

dfrazzette · 2025-11-18T05:58:10Z

Periodic jobs file updated with reporter_config list.
/pj-rehearse ack

openshift-ci-robot · 2025-11-18T05:58:13Z

@dfrazzette: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

amp-rh

Approved with minor suggested changes

ci-operator/step-registry/gs-baremetal/localnet-test/gs-baremetal-localnet-test-ref.yaml

ci-operator/step-registry/gs-baremetal/localnet-test/gs-baremetal-localnet-test-commands.sh

ci-operator/step-registry/gs-baremetal/localnet-test/README.md

ci-operator/config/RedHatQE/interop-testing/README.md

openshift-ci-robot · 2025-11-19T23:40:09Z

@dfrazzette, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

could not determine changed registry steps: could not load step registry: failed to load registry file /var/tmp/gitrepo1309303462/ci-operator/step-registry/gs-baremetal/localnet-test/gs-baremetal-localnet-test-ref.yaml: error converting YAML to JSON: yaml: line 35: did not find expected key

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

openshift-ci-robot · 2025-11-19T23:41:49Z

@dfrazzette, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

could not determine changed registry steps: could not load step registry: failed to load registry file /var/tmp/gitrepo584197868/ci-operator/step-registry/gs-baremetal/localnet-test/gs-baremetal-localnet-test-ref.yaml: error converting YAML to JSON: yaml: line 35: did not find expected key

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

openshift-ci-robot · 2025-11-20T05:33:35Z

@dfrazzette, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

could not determine changed registry steps: could not load step registry: failed to load registry file /var/tmp/gitrepo1618060347/ci-operator/step-registry/gs-baremetal/localnet-test/gs-baremetal-localnet-test-ref.yaml: error converting YAML to JSON: yaml: line 35: did not find expected key

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

openshift-ci-robot · 2025-11-20T05:50:25Z

@dfrazzette, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

could not determine changed registry steps: could not load step registry: failed to load registry file /var/tmp/gitrepo4025640195/ci-operator/step-registry/gs-baremetal/localnet-test/gs-baremetal-localnet-test-ref.yaml: error converting YAML to JSON: yaml: line 35: did not find expected key

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

ci-operator/step-registry/gs-baremetal/localnet-test/gs-baremetal-localnet-test-ref.yaml

ci-operator/config/RedHatQE/interop-testing/README.md

dfrazzette · 2025-12-23T22:26:22Z

/pj-rehearse periodic-ci-RedHatQE-interop-testing-master-gs-baremetal-localnet-ocp4.19-lp-gs-gs-baremetal-localnet-ocp419

openshift-ci-robot · 2025-12-23T22:26:24Z

@dfrazzette: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

…-metal cluster configured for Goldman Sachs.

Removed commands to fetch HCO CR and logs that do not exist.

Remove unused variable from localnet test script Remove unused variable 'hco_namespace' from script. Simplify `get storageclass` statement.

openshift-ci-robot · 2025-12-24T00:44:17Z

[REHEARSALNOTIFIER]
@dfrazzette: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
periodic-ci-RedHatQE-interop-testing-master-gs-baremetal-localnet-ocp4.19-lp-gs-gs-baremetal-localnet-ocp419	N/A	periodic	Periodic changed

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

dfrazzette · 2025-12-24T00:52:43Z

/pj-rehearse periodic-ci-RedHatQE-interop-testing-master-gs-baremetal-localnet-ocp4.19-lp-gs-gs-baremetal-localnet-ocp419

openshift-ci-robot · 2025-12-24T00:52:45Z

@dfrazzette: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

dfrazzette · 2025-12-24T01:26:40Z

/unhold

dfrazzette · 2025-12-24T01:55:27Z

The bare-metal cluster was fixed; storage class is available and ceph is healthy. However, not all OSDs are running (previously known error). After additional PR rebases, the rehearsal job did proceed past the error 'Storage Class '\''ocs-storagecluster-ceph-rbd-virtualization'\'' does not exist. Now failing due to timeouts while deleting volumesnapshots. pods not being ready.

As agreed by MPIIT management and team, since we are past the StorageClass error, we will merge this PR, and address the timeouts in a production run w/separate PR(s).
/pj-rehearse ack

openshift-ci-robot · 2025-12-24T01:55:29Z

@dfrazzette: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

Sau1506mya · 2025-12-24T14:06:31Z

/lgtm

rabin-io · 2025-12-24T17:39:13Z

/approve

openshift-ci · 2025-12-24T17:39:38Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amp-rh, dfrazzette, etirta, rabin-io, Sau1506mya

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~ci-operator/config/RedHatQE/interop-testing/OWNERS~~ [rabin-io]
~~ci-operator/jobs/RedHatQE/interop-testing/OWNERS~~ [rabin-io]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci · 2025-12-24T18:02:13Z

@dfrazzette: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/stackrox-stackrox-stackrox-stackrox-check	`0b12be8`	link	true	`/test stackrox-stackrox-stackrox-stackrox-check`
ci/rehearse/periodic-ci-RedHatQE-interop-testing-master-gs-baremetal-localnet-ocp4.19-lp-gs-gs-baremetal-localnet-ocp419	`0f77a89`	link	unknown	`/pj-rehearse periodic-ci-RedHatQE-interop-testing-master-gs-baremetal-localnet-ocp4.19-lp-gs-gs-baremetal-localnet-ocp419`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 16, 2025

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 16, 2025

openshift-ci bot requested review from dbasunag and lukas-bednar November 16, 2025 07:06

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 18, 2025

dfrazzette changed the title ~~[WIP] Create CNV scenario to run on Goldman Sachs configured bare-metal cluster.~~ Create CNV scenario to run on Goldman Sachs configured bare-metal cluster. Nov 18, 2025

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 18, 2025

openshift-ci-robot added rehearsals-ack Signifies that rehearsal jobs have been acknowledged and removed rehearsals-ack Signifies that rehearsal jobs have been acknowledged labels Nov 18, 2025

openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Nov 18, 2025

amp-rh approved these changes Nov 19, 2025

View reviewed changes

openshift-ci bot assigned amp-rh Nov 19, 2025

openshift-ci bot added lgtm Indicates that a PR is ready to be merged. and removed lgtm Indicates that a PR is ready to be merged. labels Nov 19, 2025

etirta suggested changes Nov 20, 2025

View reviewed changes

ci-operator/step-registry/gs-baremetal/localnet-test/gs-baremetal-localnet-test-ref.yaml Outdated Show resolved Hide resolved

ci-operator/config/RedHatQE/interop-testing/README.md Outdated Show resolved Hide resolved

openshift-ci bot assigned etirta Nov 20, 2025

dfrazzette force-pushed the gs_bm_cnv branch from e791410 to 0836838 Compare December 23, 2025 19:59

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Dec 23, 2025

openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Dec 23, 2025

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 23, 2025

openshift-ci-robot removed the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Dec 23, 2025

etirta and others added 5 commits December 23, 2025 19:22

Create OpenShift Virtualization scenario to run localnet test on bare…

85fa7db

…-metal cluster configured for Goldman Sachs.

Remove HCO CR and logs fetching commands.

c7b0369

Removed commands to fetch HCO CR and logs that do not exist.

Add DEFAULT_STORAGE_CLASS to localnet test config

7222c44

Remove unused variable from localnet test script Remove unused variable 'hco_namespace' from script. Simplify `get storageclass` statement.

Revert accidental changes in step-registry files; make update changes.

6dc43bf

Periodics job updated after make update.

0f77a89

dfrazzette force-pushed the gs_bm_cnv branch from 1010789 to 0f77a89 Compare December 24, 2025 00:41

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 24, 2025

openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Dec 24, 2025

openshift-ci bot assigned Sau1506mya Dec 24, 2025

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Dec 24, 2025

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 24, 2025

openshift-merge-bot bot merged commit 53e0e1f into openshift:master Dec 24, 2025
17 of 18 checks passed

dfrazzette deleted the gs_bm_cnv branch December 27, 2025 04:17

Create CNV scenario to run localnet test on Goldman Sachs configured bare-metal cluster. #71414

Create CNV scenario to run localnet test on Goldman Sachs configured bare-metal cluster. #71414

Uh oh!

Conversation

dfrazzette commented Nov 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dfrazzette commented Nov 16, 2025

Uh oh!

dfrazzette commented Nov 16, 2025

Uh oh!

openshift-ci-robot commented Nov 16, 2025

Uh oh!

dfrazzette commented Nov 16, 2025

Uh oh!

openshift-ci-robot commented Nov 16, 2025

Uh oh!

dfrazzette commented Nov 18, 2025

Uh oh!

dfrazzette commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Nov 18, 2025

Uh oh!

dfrazzette commented Nov 18, 2025

Uh oh!

openshift-ci-robot commented Nov 18, 2025

Uh oh!

amp-rh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

openshift-ci-robot commented Nov 19, 2025

Uh oh!

openshift-ci-robot commented Nov 19, 2025

Uh oh!

openshift-ci-robot commented Nov 20, 2025

Uh oh!

openshift-ci-robot commented Nov 20, 2025

Uh oh!

Uh oh!

Uh oh!

dfrazzette commented Dec 23, 2025

Uh oh!

openshift-ci-robot commented Dec 23, 2025

Uh oh!

openshift-ci-robot commented Dec 24, 2025

Uh oh!

dfrazzette commented Dec 24, 2025

Uh oh!

openshift-ci-robot commented Dec 24, 2025

Uh oh!

dfrazzette commented Dec 24, 2025

Uh oh!

dfrazzette commented Dec 24, 2025

Uh oh!

openshift-ci-robot commented Dec 24, 2025

Uh oh!

Sau1506mya commented Dec 24, 2025

Uh oh!

rabin-io commented Dec 24, 2025

Uh oh!

openshift-ci bot commented Dec 24, 2025

Uh oh!

openshift-ci bot commented Dec 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Create CNV scenario to run `localnet` test on Goldman Sachs configured bare-metal cluster. #71414

Create CNV scenario to run `localnet` test on Goldman Sachs configured bare-metal cluster. #71414

dfrazzette commented Nov 16, 2025 •

edited

Loading

dfrazzette commented Nov 18, 2025 •

edited

Loading