Refactor baremetalds-e2e-test to be more similar to openshift-e2e-test #16138

andfasano · 2021-02-22T17:07:53Z

This PR reviews deeply the baremetalds-e2e-test in order to align it, as much as possible (especially in terms of behaviors) to the openshift-e2e-test step, and more in line with the other cluster profiles. This is also part of the ongoing effort to expand the OpenShift e2e CI coverage for the baremetal ipi platform, so that it could be possible to apply the whole openshift/conformance suite.

Relevant changes introduced:

The baremetal platform requires the tests image mirroring feature (see [1]) to execute some of the e2e tests.
- Dev-scripts private registry has been used to store the mirrored test images.
- Test images mirroring is currently available only in the openshift-tests binary for OpenShift 4.7 or greater. To maintain the backward compatibility, for jobs on branches < 4.7 a minimal list is adopted (defined by TEST_MINIMAL_LIST)
RUN_UPGRADE_TEST has been deprecated
baremetalds-e2e-conf step is not used anymore
TEST_SKIPS is currently using to exclude temporarily those tests that are failing for which an analysis/fix is in progress

[1] openshift/origin#24887

andfasano · 2021-02-23T15:12:44Z

/test pj-rehearse

andfasano · 2021-02-25T15:49:07Z

/test pj-rehearse

andfasano · 2021-03-01T08:48:58Z

/test pj-rehearse

andfasano · 2021-03-01T14:30:41Z

/test pj-rehearse

andfasano · 2021-03-01T14:55:16Z

/test pj-rehearse

andfasano · 2021-03-02T18:02:12Z

/test pj-rehearse

ardaguclu · 2021-03-11T11:33:34Z

ci-operator/step-registry/baremetalds/e2e/test/baremetalds-e2e-test-commands.sh

Is this timeout usage valid for the 'standard' workflows. Because I saw a couple of different places have different timeout values being set in this PR.

No, it's a workaround added to gather some results when a test get stuck, hopefully it could be removed once the test failures will get fixed (and it should be properly sized to give enough time to the tests to complete in the standard case).

andfasano · 2021-03-11T12:03:59Z

/test pj-rehearse

honza · 2021-03-11T13:24:37Z

Please include the PR description in the commit message

andfasano · 2021-03-11T16:30:29Z

/test pj-rehearse

andfasano · 2021-03-12T09:18:55Z

/test pj-rehearse

andfasano · 2021-03-12T14:32:02Z

/test pj-rehearse

andfasano · 2021-03-12T15:45:05Z

/test pj-rehearse

andfasano · 2021-03-15T09:53:27Z

/test pj-rehearse

andfasano · 2021-03-15T16:17:33Z

/test pj-rehearse

andfasano · 2021-03-15T18:56:30Z

/test pj-rehearse

andfasano · 2021-03-16T18:06:14Z

/test pj-rehearse

openshift-ci · 2021-03-17T10:55:26Z

@andfasano: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/rehearse/openshift/cluster-baremetal-operator/release-4.6/e2e-metal-ipi	8de04484d3e39663b05fd972b013f1f2d206f4a0	link	`/test pj-rehearse`
ci/rehearse/openshift/cluster-api-provider-baremetal/master/e2e-metal-ipi-upgrade	f8341aae1fd950cb70f37a43b1c98feecb417486	link	`/test pj-rehearse`
ci/rehearse/openshift/baremetal-operator/master/e2e-metal-ipi-trigger	f8341aae1fd950cb70f37a43b1c98feecb417486	link	`/test pj-rehearse`
ci/prow/pj-rehearse	f8341aae1fd950cb70f37a43b1c98feecb417486	link	`/test pj-rehearse`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

This commits modifies baremetalds-e2e-test in order to align it, as much as possible (especially in terms of behaviors) to the openshift-e2e-test step, and more in line with the other cluster profiles. This is also part of the ongoing effort to expand the OpenShift e2e CI coverage for the baremetal ipi platform, so that it could be possible to apply the whole openshift/conformance suite. Relevant changes introduced: - The baremetal platform requires the tests image mirroring feature (see [1]) to execute some of the e2e tests. -- Dev-scripts private registry has been used to store the mirrored test images. -- Test images mirroring is currently available only in the openshift-tests binary for OpenShift 4.7 or greater. To maintain the backward compatibility, for jobs on branches < 4.7 a minimal list is adopted (defined by TEST_MINIMAL_LIST) - RUN_UPGRADE_TEST has been deprecated - baremetalds-e2e-conf step is not used anymore - TEST_SKIPS is currently using to exclude temporarily those tests that are failing for which an analysis/fix is in progress

honza · 2021-03-17T13:19:19Z

/lgtm

openshift-ci-robot · 2021-03-17T13:19:40Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andfasano, honza

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~ci-operator/step-registry/baremetalds/OWNERS~~ [andfasano,honza]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2021-03-17T13:34:03Z

@andfasano: Updated the following 2 configmaps:

step-registry configmap in namespace ci at cluster app.ci using the following files:
- key baremetalds-devscripts-setup-commands.sh using file ci-operator/step-registry/baremetalds/devscripts/setup/baremetalds-devscripts-setup-commands.sh
- key baremetalds-e2e-test-commands.sh using file ci-operator/step-registry/baremetalds/e2e/test/baremetalds-e2e-test-commands.sh
- key baremetalds-e2e-test-ref.yaml using file ci-operator/step-registry/baremetalds/e2e/test/baremetalds-e2e-test-ref.yaml
- key baremetalds-e2e-upgrade-workflow.yaml using file ci-operator/step-registry/baremetalds/e2e/upgrade/baremetalds-e2e-upgrade-workflow.yaml
step-registry configmap in namespace ci at cluster api.ci using the following files:
- key baremetalds-devscripts-setup-commands.sh using file ci-operator/step-registry/baremetalds/devscripts/setup/baremetalds-devscripts-setup-commands.sh
- key baremetalds-e2e-test-commands.sh using file ci-operator/step-registry/baremetalds/e2e/test/baremetalds-e2e-test-commands.sh
- key baremetalds-e2e-test-ref.yaml using file ci-operator/step-registry/baremetalds/e2e/test/baremetalds-e2e-test-ref.yaml
- key baremetalds-e2e-upgrade-workflow.yaml using file ci-operator/step-registry/baremetalds/e2e/upgrade/baremetalds-e2e-upgrade-workflow.yaml

Details

In response to this:

This PR reviews deeply the baremetalds-e2e-test in order to align it, as much as possible (especially in terms of behaviors) to the openshift-e2e-test step, and more in line with the other cluster profiles. This is also part of the ongoing effort to expand the OpenShift e2e CI coverage for the baremetal ipi platform, so that it could be possible to apply the whole openshift/conformance suite.

Relevant changes introduced:

The baremetal platform requires the tests image mirroring feature (see [1]) to execute some of the e2e tests.

Dev-scripts private registry has been used to store the mirrored test images.

Test images mirroring is currently available only in the openshift-tests binary for OpenShift 4.7 or greater. To maintain the backward compatibility, for jobs on branches < 4.7 a minimal list is adopted (defined by TEST_MINIMAL_LIST)

RUN_UPGRADE_TEST has been deprecated

baremetalds-e2e-conf step is not used anymore

TEST_SKIPS is currently using to exclude temporarily those tests that are failing for which an analysis/fix is in progress

[1] openshift/origin#24887

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 22, 2021

openshift-ci-robot requested review from akiselev1 and honza February 22, 2021 17:09

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 22, 2021

andfasano force-pushed the metal-align-e2e-test branch from df4d814 to 10441f7 Compare February 22, 2021 17:41

andfasano force-pushed the metal-align-e2e-test branch 6 times, most recently from 370c30f to ab3af77 Compare February 25, 2021 12:35

andfasano force-pushed the metal-align-e2e-test branch from ab3af77 to 460157f Compare February 25, 2021 17:22

andfasano force-pushed the metal-align-e2e-test branch 2 times, most recently from ea7d3d7 to e54dc41 Compare March 1, 2021 12:59

andfasano force-pushed the metal-align-e2e-test branch 2 times, most recently from c1b42d2 to afc8cba Compare March 2, 2021 11:05

openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 2, 2021

andfasano force-pushed the metal-align-e2e-test branch from afc8cba to 435e518 Compare March 2, 2021 11:30

openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 2, 2021

andfasano force-pushed the metal-align-e2e-test branch 2 times, most recently from 2f3ae98 to b339929 Compare March 2, 2021 15:49

andfasano force-pushed the metal-align-e2e-test branch 3 times, most recently from 7f80b89 to a50b274 Compare March 3, 2021 14:50

ardaguclu reviewed Mar 11, 2021

View reviewed changes

andfasano force-pushed the metal-align-e2e-test branch from 7286369 to 55c0214 Compare March 11, 2021 15:09

andfasano force-pushed the metal-align-e2e-test branch from ecf5bb1 to 278b63e Compare March 12, 2021 08:15

andfasano force-pushed the metal-align-e2e-test branch from 278b63e to 8de0448 Compare March 15, 2021 08:56

andfasano force-pushed the metal-align-e2e-test branch 3 times, most recently from 30a4990 to 489b22f Compare March 16, 2021 15:16

andfasano force-pushed the metal-align-e2e-test branch 2 times, most recently from e15ecb2 to 2e098fc Compare March 17, 2021 11:51

andfasano force-pushed the metal-align-e2e-test branch from 2e098fc to fecff22 Compare March 17, 2021 12:45

andfasano changed the title ~~[WIP] Refactor baremetalds-e2e-test to be more similar to openshift-e2e-test~~ Refactor baremetalds-e2e-test to be more similar to openshift-e2e-test Mar 17, 2021

openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 17, 2021

openshift-ci-robot assigned honza Mar 17, 2021

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Mar 17, 2021

openshift-merge-robot merged commit 6b3e263 into openshift:master Mar 17, 2021

Refactor baremetalds-e2e-test to be more similar to openshift-e2e-test #16138

Refactor baremetalds-e2e-test to be more similar to openshift-e2e-test #16138

Uh oh!

Conversation

andfasano commented Feb 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andfasano commented Feb 23, 2021

Uh oh!

andfasano commented Feb 25, 2021

Uh oh!

andfasano commented Mar 1, 2021

Uh oh!

andfasano commented Mar 1, 2021

Uh oh!

andfasano commented Mar 1, 2021

Uh oh!

andfasano commented Mar 2, 2021

Uh oh!

ardaguclu Mar 11, 2021

Choose a reason for hiding this comment

Uh oh!

andfasano Mar 11, 2021

Choose a reason for hiding this comment

Uh oh!

andfasano commented Mar 11, 2021

Uh oh!

honza commented Mar 11, 2021

Uh oh!

andfasano commented Mar 11, 2021

Uh oh!

andfasano commented Mar 12, 2021

Uh oh!

andfasano commented Mar 12, 2021

Uh oh!

andfasano commented Mar 12, 2021

Uh oh!

andfasano commented Mar 15, 2021

Uh oh!

andfasano commented Mar 15, 2021

Uh oh!

andfasano commented Mar 15, 2021

Uh oh!

andfasano commented Mar 16, 2021

Uh oh!

openshift-ci bot commented Mar 17, 2021

Uh oh!

honza commented Mar 17, 2021

Uh oh!

openshift-ci-robot commented Mar 17, 2021

Uh oh!

openshift-ci-robot commented Mar 17, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

andfasano commented Feb 22, 2021 •

edited

Loading