Skip to content

Conversation

@danielerez
Copy link
Contributor

Added support for running the agent flow using the openshift-appliance[1] disk image.
To invoke the flow, use the following env var in config:
export AGENT_E2E_TEST_BOOT_MODE=DISKIMAGE

The appliance flow is as follows:

  • Generate appliance-config.yaml[2]
    • Includes config required for building the appliance
  • Build the appliance disk image using the container image defined in APPLIANCE_IMAGE env var
  • Create the config-image ISO
  • Clone the appliance disk image for every node
    • As each node needs a seperate disk for exclusive i/o
  • Attach a cloned disk image to each node
  • Attach the config-image ISO to all nodes
  • Boot all machines from the disk image

Note: based on config-image support PR

[1] https://github.com/openshift/appliance
[2] https://github.com/openshift/appliance/blob/master/docs/user-guide.md#set-appliance-config

@openshift-ci openshift-ci bot requested review from bfournie and sadasu July 6, 2023 10:09
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 6, 2023
@openshift-ci openshift-ci bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jul 6, 2023
@openshift-ci
Copy link

openshift-ci bot commented Jul 6, 2023

Hi @danielerez. Thanks for your PR.

I'm waiting for a openshift-metal3 member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 6, 2023
@danielerez
Copy link
Contributor Author

/cc @rwsu
/cc @andfasano

@openshift-ci openshift-ci bot requested review from andfasano and rwsu July 10, 2023 17:13
@danielerez danielerez changed the title [WIP] agent: introduce openshift-appliance flow agent: introduce openshift-appliance flow Jul 12, 2023
@andfasano
Copy link
Member

/ok-to-test

@openshift-ci openshift-ci bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 12, 2023
@andfasano
Copy link
Member

@danielerez can you please share the min config to try it out? I'm using:

export OPENSHIFT_RELEASE_IMAGE=registry.ci.openshift.org/ocp/release:4.14.0-0.nightly-2023-07-05-191022
export AGENT_E2E_TEST_SCENARIO=COMPACT_IPV4
export AGENT_E2E_TEST_BOOT_MODE=DISKIMAGE

but got a failure when running the openshift-appliance image:

...
+(./agent/06_agent_create_cluster.sh:330): create_appliance(): local asset_dir=/home/afasano/dev-scripts/ocp/ostest
+(./agent/06_agent_create_cluster.sh:334): create_appliance(): sudo podman run -it --rm --privileged --net=host -v /home/afasano/dev-scripts/ocp/ostest:/assets:Z quay.io/edge-infrastructure/openshift-appliance:latest build --debug-base-ignition
Trying to pull quay.io/edge-infrastructure/openshift-appliance:latest...
Getting image source signatures
Copying blob sha256:6dcc6fe5530e8af484354433fbe3cb2001508b656a4c9ddd86eea7f0d1048d52
Copying blob sha256:f3a0532868dc138537e58032870b334967dd4df98206d907523a62598ddde8ff
Copying config sha256:3f6f1370a189932046547de2aee699bc38cba16bc202b0511f1b96c162c7fa70
Writing manifest to image destination
Storing signatures
FATAL failed to fetch Appliance disk image: failed to fetch dependency of "Appliance disk image": failed to fetch dependency of "Appliance Recovery ISO": failed to generate asset "Recovery Ignition": failed to fetch un-configured ignition: open assets/temp/unconfigured-agent.ign: no such file or directory 

common.sh Outdated
if [[ ! -z ${AGENT_E2E_TEST_BOOT_MODE} ]]; then
if [[ $AGENT_E2E_TEST_BOOT_MODE != "ISO" && $AGENT_E2E_TEST_BOOT_MODE != "PXE" ]]; then
if [[ $AGENT_E2E_TEST_BOOT_MODE != "ISO" && $AGENT_E2E_TEST_BOOT_MODE != "PXE" && $AGENT_E2E_TEST_BOOT_MODE != "DISKIMAGE" ]]; then
printf "Found invalid value \"$AGENT_E2E_TEST_BOOT_MODE\" for AGENT_E2E_TEST_BOOT_MODE. Supported values: ISO (default), PXE."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
printf "Found invalid value \"$AGENT_E2E_TEST_BOOT_MODE\" for AGENT_E2E_TEST_BOOT_MODE. Supported values: ISO (default), PXE."
printf "Found invalid value \"$AGENT_E2E_TEST_BOOT_MODE\" for AGENT_E2E_TEST_BOOT_MODE. Supported values: ISO (default), PXE, DISKIMAGE."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

# This is required to allow qemu opening the disk image
if [ "${OPENSHIFT_CI}" == true ]; then
setfacl -m u:qemu:rx /root
fi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a certain amount of duplicated code with attach_agent_iso(), it'd be nice to remove it where possible, ie like this block

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it could be pretty cumbersome to add this logic to 'attach_agent_iso' func. Seems it has a substantial different behaviour than in the appliance flow. But I'll indeed try to extract the duplicated code where possible.

# Attach the appliance disk image and the config ISO
sudo virt-xml ${name} --remove-device --disk 1
sudo virt-xml ${name} --add-device --disk "${disk_image}",device=disk,target.dev=sda
sudo virt-xml ${name} --add-device --disk "${config_image_dir}/agentconfig.noarch.iso",device=cdrom,target.dev=${config_image_drive}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

additional spaces?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably.. fixed.


# Attach the appliance disk image and the config ISO
sudo virt-xml ${name} --remove-device --disk 1
sudo virt-xml ${name} --add-device --disk "${disk_image}",device=disk,target.dev=sda
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q: what's the reason to remove and re-add the disk?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we don't need the default disk attached to the machine, as we're using our own generated disk image. I.e. need to replace the existing disk with our own.


}

function attach_appliance_diskimage() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this function populates the $OCP_DIR with several artifacts, but the make clean is broken as the cache and temp folder are created under the root user. Please add the required cleaning steps so that it would be possible to remove everything when running the clean target

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, good catch, will add.

ocpRelease:
version: {{ version }}
channel: candidate
cpuArchitecture: x86_64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the appliance support arm64? A recent PR brought in support for arm64: #1518

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. We haven't actually tested yet other architecture, but the appliance should support it. We can make it configurable for later on then.

version: {{ version }}
channel: candidate
cpuArchitecture: x86_64
diskSizeGB: 200
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the diskSizeGB be set to a smaller size? 200G per VM fills up the disk on my local test machine. What would be a recommended minimum size?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's actaully only the virtual size of the disk, so it shouldn't consume that much space of disk. The actual size is ~40GiB.

@danielerez
Copy link
Contributor Author

danielerez commented Jul 14, 2023

@danielerez can you please share the min config to try it out? I'm using:

export OPENSHIFT_RELEASE_IMAGE=registry.ci.openshift.org/ocp/release:4.14.0-0.nightly-2023-07-05-191022
export AGENT_E2E_TEST_SCENARIO=COMPACT_IPV4
export AGENT_E2E_TEST_BOOT_MODE=DISKIMAGE

but got a failure when running the openshift-appliance image:

...
+(./agent/06_agent_create_cluster.sh:330): create_appliance(): local asset_dir=/home/afasano/dev-scripts/ocp/ostest
+(./agent/06_agent_create_cluster.sh:334): create_appliance(): sudo podman run -it --rm --privileged --net=host -v /home/afasano/dev-scripts/ocp/ostest:/assets:Z quay.io/edge-infrastructure/openshift-appliance:latest build --debug-base-ignition
Trying to pull quay.io/edge-infrastructure/openshift-appliance:latest...
Getting image source signatures
Copying blob sha256:6dcc6fe5530e8af484354433fbe3cb2001508b656a4c9ddd86eea7f0d1048d52
Copying blob sha256:f3a0532868dc138537e58032870b334967dd4df98206d907523a62598ddde8ff
Copying config sha256:3f6f1370a189932046547de2aee699bc38cba16bc202b0511f1b96c162c7fa70
Writing manifest to image destination
Storing signatures
FATAL failed to fetch Appliance disk image: failed to fetch dependency of "Appliance disk image": failed to fetch dependency of "Appliance Recovery ISO": failed to generate asset "Recovery Ignition": failed to fetch un-configured ignition: open assets/temp/unconfigured-agent.ign: no such file or directory 

Looks like the unconfigured-ignition API is missing from the openshift-install binary.
So I actually using a custom build of openshift-install for now, or enable building the openshift-install binary from master:
I.e. export KNI_INSTALL_FROM_GIT=true
Maybe the used nightly don't include that API :/

@bfournie
Copy link
Contributor

For builds after 7/12 you should be able to get a valid unconfigured ignition and config image built. For example using these values:
export AGENT_USE_APPLIANCE_MODEL=true
export AGENT_APPLIANCE_HOTPLUG=true
export AGENT_E2E_TEST_SCENARIO=COMPACT_IPV4

agent_nodes_hostnames: "{{ lookup('env', 'AGENT_NODES_HOSTNAMES_STR') }}"
agent_use_ztp_manifests: "{{ lookup('env', 'AGENT_USE_ZTP_MANIFESTS') }}"
agent_test_cases: "{{ lookup('env', 'AGENT_TEST_CASES') }}"
agent_use_appliance_manifest: "{{ lookup('env', 'AGENT_USE_APPLIANCE_MANIFEST') }}"
Copy link
Contributor

@bfournie bfournie Jul 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can AGENT_E2E_TEST_BOOT_MODE be used here instead of adding an additional AGENT_USE_APPLIANCE_MANIFEST export?

If its really necessary to add an additional export will need to add it to config_example.sh

Copy link
Contributor Author

@danielerez danielerez Jul 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually just wanted to be explicit here, for keeping the convention of other manifests (e.g. AGENT_USE_ZTP_MANIFESTS). We can indeed use AGENT_E2E_TEST_BOOT_MODE if you think it'd be clearer(?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can use AGENT_E2E_TEST_BOOT_MODE to simplify the number of exports

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'll add it to vars/main.yml then.

ipxeBaseURL: {{ pxe_server_url }}
{% endif %}
{% if networking_mode != "dhcp" %}
{% if networking_mode != "DHCP" %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be able to rebase and pick up this fix.

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 18, 2023
@danielerez danielerez force-pushed the appliance-flow branch 2 times, most recently from 9a2bd28 to 2c68bfa Compare July 18, 2023 08:41
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 18, 2023
if [[ $AGENT_E2E_TEST_BOOT_MODE != "ISO" && $AGENT_E2E_TEST_BOOT_MODE != "PXE" ]]; then
printf "Found invalid value \"$AGENT_E2E_TEST_BOOT_MODE\" for AGENT_E2E_TEST_BOOT_MODE. Supported values: ISO (default), PXE."
if [[ $AGENT_E2E_TEST_BOOT_MODE != "ISO" && $AGENT_E2E_TEST_BOOT_MODE != "PXE" && $AGENT_E2E_TEST_BOOT_MODE != "DISKIMAGE" ]]; then
printf "Found invalid value \"$AGENT_E2E_TEST_BOOT_MODE\" for AGENT_E2E_TEST_BOOT_MODE. Supported values: ISO (default), PXE, DISKIMAGE."
Copy link
Member

@andfasano andfasano Jul 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: feeling that a case may be a little bit more readable with a longer list, ie something like

case "$AGENT_E2E_TEST_BOOT_MODE" in
  "ISO" | "PXE" | "DISKIMAGE")
    # Valid value
    ;;
  *)
    printf "Found invalid value \"$AGENT_E2E_TEST_BOOT_MODE\" for AGENT_E2E_TEST_BOOT_MODE. Supported values: ISO (default), PXE, DISKIMAGE."
    exit 1
    ;;
esac

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea:) Added.

@bfournie
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 19, 2023
@bfournie
Copy link
Contributor

/retest-required

Copy link
Contributor

@rwsu rwsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @danielerez, I'm having trouble testing this PR. It is not clear to me how to set the release image version.
When I have export OPENSHIFT_RELEASE_IMAGE=quay.io/openshift-release-dev/ocp-release:4.14.0-ec.3-x86_64 in config_x.sh I see:

Jul 26 21:44:38 master-0 load-config-iso.sh[5429]: <  releaseImage: quay.io/openshift-release-dev/ocp-release@sha256:3c050cb52fdd3e65c518d4999d238ec026ef724503f275377fee6bf0d33093ab
Jul 26 21:44:38 master-0 load-config-iso.sh[5429]: ---
Jul 26 21:44:38 master-0 load-config-iso.sh[5429]: >  releaseImage: quay.io/openshift-release-dev/ocp-release:4.14.0-ec.3-x86_64
Jul 26 21:44:38 master-0 load-config-iso.sh[1535]: The cluster-image-set in archive does not match current release quay.io/openshift-release-dev/ocp-release@sha256:3c050cb52fdd3e65c518d4999d238ec026ef724503f275377fee6bf0d33093ab

Not setting any release image version, yields similar results.

for (( n=0; n<${2}; n++ ))
do
name=${CLUSTER_NAME}_${1}_${n}
disk_image=${appliance_disk_image}_${n}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
disk_image=${appliance_disk_image}_${n}
disk_image=${appliance_disk_image}_${1}_${n}

When there are workers defined in the cluster. There is an issue where master-0's disk image name and worker-0's disk image name are the same causing a write lock error:

+(./agent/06_agent_create_cluster.sh:73): create_config_image(): cp -r ocp/ostest/configimage/auth ocp/ostest
+(./agent/06_agent_create_cluster.sh:130): attach_appliance_diskimage(): (( n=0 ))
+(./agent/06_agent_create_cluster.sh:130): attach_appliance_diskimage(): (( n<2 ))
+(./agent/06_agent_create_cluster.sh:132): attach_appliance_diskimage(): name=ostest_worker_0
+(./agent/06_agent_create_cluster.sh:133): attach_appliance_diskimage(): disk_image=ocp/ostest/appliance.raw_0
+(./agent/06_agent_create_cluster.sh:136): attach_appliance_diskimage(): sudo cp ocp/ostest/appliance.raw ocp/ostest/appliance.raw_0
+(./agent/06_agent_create_cluster.sh:139): attach_appliance_diskimage(): sudo virt-xml ostest_worker_0 --remove-device --disk 1
Domain 'ostest_worker_0' defined successfully.
+(./agent/06_agent_create_cluster.sh:140): attach_appliance_diskimage(): sudo virt-xml ostest_worker_0 --add-device --disk ocp/ostest/appliance.raw_0,device=disk,target.dev=sda
Domain 'ostest_worker_0' defined successfully.
+(./agent/06_agent_create_cluster.sh:141): attach_appliance_diskimage(): sudo virt-xml ostest_worker_0 --add-device --disk ocp/ostest/configimage/agentconfig.noarch.iso,device=cdrom,target.dev=sdd
Domain 'ostest_worker_0' defined successfully.
+(./agent/06_agent_create_cluster.sh:144): attach_appliance_diskimage(): sudo virt-xml ostest_worker_0 --edit target=sda --disk=boot_order=1 --start
ERROR    Failed starting domain 'ostest_worker_0': internal error: qemu unexpectedly closed the monitor: 2023-07-26T21:10:15.312697Z qemu-kvm: -device ide-hd,bus=ide.0,drive=libvirt-2-format,id=sata0-0-0,bootindex=1: Failed to get "write" lock
Is another process using the image [/home/rwsu/go/src/github.com/openshift-metal3/dev-scripts/ocp/ostest/appliance.raw_0]?
Domain 'ostest_worker_0' defined successfully.

I think the disk_image name should include the "master" or "worker" substring.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, missed it, good catch. Fixed.

@danielerez
Copy link
Contributor Author

Hi @danielerez, I'm having trouble testing this PR. It is not clear to me how to set the release image version. When I have export OPENSHIFT_RELEASE_IMAGE=quay.io/openshift-release-dev/ocp-release:4.14.0-ec.3-x86_64 in config_x.sh I see:

Jul 26 21:44:38 master-0 load-config-iso.sh[5429]: <  releaseImage: quay.io/openshift-release-dev/ocp-release@sha256:3c050cb52fdd3e65c518d4999d238ec026ef724503f275377fee6bf0d33093ab
Jul 26 21:44:38 master-0 load-config-iso.sh[5429]: ---
Jul 26 21:44:38 master-0 load-config-iso.sh[5429]: >  releaseImage: quay.io/openshift-release-dev/ocp-release:4.14.0-ec.3-x86_64
Jul 26 21:44:38 master-0 load-config-iso.sh[1535]: The cluster-image-set in archive does not match current release quay.io/openshift-release-dev/ocp-release@sha256:3c050cb52fdd3e65c518d4999d238ec026ef724503f275377fee6bf0d33093ab

Not setting any release image version, yields similar results.

This issue was actually fixed about a week ago. You might have a stale image of the appliance.
I've added to agent/cleanup.sh a step for removing the the image.
Can you please run 'make clean' and try again?

I've just used the following configuration for running the flow:

export AGENT_E2E_TEST_SCENARIO=HA_IPV4
export AGENT_E2E_TEST_BOOT_MODE=DISKIMAGE
make clean && make run

@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Jul 27, 2023

# Build appliance with `debug-base-ignition` flag for using the custom openshift-install
# binary from assets directory.
sudo podman run -it --rm --privileged --net=host -v ${asset_dir}:/assets:Z ${APPLIANCE_IMAGE} build --debug-base-ignition
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
sudo podman run -it --rm --privileged --net=host -v ${asset_dir}:/assets:Z ${APPLIANCE_IMAGE} build --debug-base-ignition
sudo podman run -it --rm --pull newer --privileged --net=host -v ${asset_dir}:/assets:Z ${APPLIANCE_IMAGE} build --debug-base-ignition

Can we have podman pull a newer version of one is available?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, good idea, added.

@rwsu
Copy link
Contributor

rwsu commented Jul 27, 2023

I've just used the following configuration for running the flow:

export AGENT_E2E_TEST_SCENARIO=HA_IPV4
export AGENT_E2E_TEST_BOOT_MODE=DISKIMAGE
make clean && make run

This didn't work for me after I pulled a newer appliance image.

Jul 27 20:48:15 master-0 load-config-iso.sh[1558]: <   releaseImage: registry.ci.openshift.org/origin/release:4.14
Jul 27 20:48:15 master-0 load-config-iso.sh[1558]: ---
Jul 27 20:48:15 master-0 load-config-iso.sh[1558]: >   releaseImage: registry.ci.openshift.org/ocp/release:4.14.0-0.nightly-2023-07-27-104118

I had to manually set OPENSHIFT_RELEASE_IMAGE=registry.ci.openshift.org/origin/release:4.14.

Added support for running the agent flow using the
openshift-appliance[1] disk image.

To invoke the flow, use the following env var in config:
export AGENT_E2E_TEST_BOOT_MODE=DISKIMAGE

The appliance flow is as follows:
* Generate appliance-config.yaml[2]
  * Includes config required for building the appliance
* Build the appliance disk image using the container
  image defined in APPLIANCE_IMAGE env var
* Create the config-image ISO
* Clone the appliance disk image for every node
  * As each node needs a seperate disk for exclusive i/o
* Attach a cloned disk image to each node
* Attach the config-image ISO to all nodes
* Boot all machines from the disk image

Note: based on config-image support PR[3]

[1] https://github.com/openshift/appliance
[2] https://github.com/openshift/appliance/blob/master/docs/user-guide.md#set-appliance-config
[3] openshift-metal3#1533
@danielerez
Copy link
Contributor Author

I've just used the following configuration for running the flow:
export AGENT_E2E_TEST_SCENARIO=HA_IPV4
export AGENT_E2E_TEST_BOOT_MODE=DISKIMAGE
make clean && make run

This didn't work for me after I pulled a newer appliance image.

Jul 27 20:48:15 master-0 load-config-iso.sh[1558]: <   releaseImage: registry.ci.openshift.org/origin/release:4.14
Jul 27 20:48:15 master-0 load-config-iso.sh[1558]: ---
Jul 27 20:48:15 master-0 load-config-iso.sh[1558]: >   releaseImage: registry.ci.openshift.org/ocp/release:4.14.0-0.nightly-2023-07-27-104118

I had to manually set OPENSHIFT_RELEASE_IMAGE=registry.ci.openshift.org/origin/release:4.14.

I wonder what's the difference in the envs then:/ Any way, we'll have to test it on the actual CI env...

@danielerez
Copy link
Contributor Author

Hey @rwsu @andfasano @bfournie, can you please take another look? I think this change is ready for approval now.
We need to merge it for testing on CI machines... So further fixes if needed would probably be done in the appliance.

Copy link
Contributor

@rwsu rwsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
Not sure how CI will be configured to avoid the release image mismatch problem. Other than that, it looks ok to me.

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 1, 2023
@bfournie
Copy link
Contributor

bfournie commented Aug 1, 2023

/approve

@openshift-ci
Copy link

openshift-ci bot commented Aug 1, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bfournie

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 1, 2023
@danielerez
Copy link
Contributor Author

/test e2e-metal-ipi-serial-ipv4

@andfasano
Copy link
Member

/lgtm

@danielerez
Copy link
Contributor Author

/test e2e-metal-ipi-serial-ipv4

3 similar comments
@danielerez
Copy link
Contributor Author

/test e2e-metal-ipi-serial-ipv4

@danielerez
Copy link
Contributor Author

/test e2e-metal-ipi-serial-ipv4

@danielerez
Copy link
Contributor Author

/test e2e-metal-ipi-serial-ipv4

@danielerez
Copy link
Contributor Author

/retest

@openshift-ci
Copy link

openshift-ci bot commented Aug 3, 2023

@danielerez: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-agent-ha-dualstack 7e3d43e link false /test e2e-agent-ha-dualstack

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@danielerez
Copy link
Contributor Author

/retest-required

@openshift-merge-robot openshift-merge-robot merged commit ff55469 into openshift-metal3:master Aug 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants