AGENT-724: Deploy vSphere clusters with credentials by rwsu · Pull Request #1599 · openshift-metal3/dev-scripts

rwsu · 2023-11-02T20:34:34Z

Fake credentials are added to install-config.yaml for vSphere platform.

In combination with
#1588, this patch allows testng of installer and assisted-service validations for the vSphere platform.

A complete install for vSphere using dev-scripts is currently not possible because the vSphere environment is partially simulated using libvirt VMs. Any cloud controller manager interactions with vSphere will fail in libvirt because there isn't an actual vSphere environment that is being exercised.

In this setup, only "agent wait-for bootstrap-complete" will succeed. For vSphere, "agent wait-for install-complete" will fail and is skipped.

Add these to config_.sh script to test:

export AGENT_PLATFORM_TYPE=vsphere
export AGENT_TEST_CASES=skip_wait_for_install_complete

rwsu · 2023-11-02T20:35:20Z

/cc @bfournie @andfasano
/uncc @lranjbar

bfournie · 2023-11-09T18:35:27Z

    platformName: oci
 {% elif (platform_type == "vsphere") %}
  vsphere:
+    vcenters:


Now that there is more configuration for vsphere, recommend adding a new file install-config_vsphere_yaml.j2 and using it here similar to what we did for baremetal, based on @andfasano' suggestion - https://github.com/openshift-metal3/dev-scripts/blob/master/agent/roles/manifests/tasks/install-config.yml#L12-L16

At some point though we'll have to make macros out of the common areas in these platform specific install-config files

I created https://issues.redhat.com/browse/AGENT-746 to add macros to these platform specific install-config files.

bfournie · 2023-11-09T18:36:17Z

      exit 1
  fi

+  if [[ $AGENT_TEST_CASES =~ "skip_wait_for_install_complete" ]]; then


Would it make sense to set this automatically when the platform=vsphere?

That's what I had it set to originally :). But then though to have it independent of platform type in the event we eventually fully support the vsphere environment. I'm ok either way.

OK that's fine leaving it as independent. I remembered that external platform can probably use this too.

TBTH this doesn't seem to me a real test case: it doesn't exercise the environment nor perform any verification, and it's not used in run_agent_test_cases (and IIUC we're not even performing a complete installation). So, as long as it is useful to verify a part of the agent vsphere workflow, it could be managed via a normal config var, ie WAIT_FOR_INSTALL_COMPLETE=false

Thanks Andrea. I've switch to using a normal config var WAIT_FOR_INSTALL_COMPLETE as you suggested.

rwsu · 2023-11-09T18:53:08Z

/test e2e-agent-compact-ipv4
Last failure:

level=debug msg=* Could not update servicemonitor "openshift-machine-config-operator/machine-config-daemon" (830 of 861)
level=debug msg=* Could not update servicemonitor "openshift-operator-lifecycle-manager/olm-operator" (839 of 861)
level=debug msg=* Could not update servicemonitor "openshift-service-ca-operator/service-ca-operator" (861 of 861)
level=debug msg=Still waiting for the cluster to initialize: Working towards 4.14.0-0.ci.test-2023-11-08-231053-ci-op-8v0g2rn8-latest: 541 of 861 done (62% complete)
level=debug msg=Still waiting for the cluster to initialize: Cluster operators authentication, kube-apiserver are not available
level=debug msg=Still waiting for the cluster to initialize: Cluster operator authentication is not available
level=debug msg=Still waiting for the cluster to initialize: Cluster operator etcd is degraded
level=debug msg=Still waiting for the cluster to initialize: Cluster operator etcd is degraded
bash: line 190: 17370 Killed                  timeout -s 9 105m make agent
Fetching kubeconfig, other credentials...
Adding proxy-url in kubeconfig
dev-scripts setup completed, fetching logs
tar: Removing leading `/' from member names
Removing REDACTED info from log...
{"component":"entrypoint","error":"wrapped process failed: exit status 137","file":"k8s.io/test-infra/prow/entrypoint/run.go:84","func":"k8s.io/test-infra/prow/entrypoint.Options.internalRun","level":"error","msg":"Error executing test process","severity":"error","time":"2023-11-09T01:01:52Z"}
error: failed to execute wrapped command: exit status 137

The job runs the compact_ipv4 scenario for the vsphere platform. The "agent wait-for install-complete" step is skipped because install-complete does not succeed for vsphere using libvirt virtual machines. Depends on: openshift-metal3/dev-scripts#1599 Signed-off-by: Richard Su <rwsu@redhat.com>

bfournie · 2023-11-10T14:51:56Z

/approve

openshift-ci · 2023-11-10T14:52:03Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bfournie

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [bfournie]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

rwsu · 2023-11-10T18:06:03Z

-{% elif (platform_type == "vsphere") %}
-  vsphere:
-{% set macs = agent_nodes_macs.split(',') %}
-{% set hostnames = agent_nodes_hostnames.split(',') %}


Removed unused macs and hostnames fields.

The job runs the compact_ipv4 scenario for the vsphere platform. The "agent wait-for install-complete" step is skipped because install-complete does not succeed for vsphere using libvirt virtual machines. Depends on: openshift-metal3/dev-scripts#1599 Signed-off-by: Richard Su <rwsu@redhat.com>

andfasano · 2024-02-22T10:31:35Z

      exit 1
  fi

+  if [ "${WAIT_FOR_INSTALL_COMPLETE}" == "false" ]; then


Is that meant to be agent only? In such case I think it would be more readable to prefix wiht AGENT_, otherwise if it's a general config var it should be applied also in the non-agent workflow?

It is agent only. I've renamed it to AGENT_WAIT_FOR_INSTALL_COMPLETE.

Fake credentials are added to install-config.yaml for vSphere platform. In combination with openshift-metal3#1588, this patch allows testng of installer and assisted-service validations for the vSphere platform. A complete install for vSphere using dev-scripts is currently not possible because the vSphere environment is partially simulated using libvirt VMs. Any cloud controller manager interactions with vSphere will fail in libvirt because there isn't an actual vSphere environment that is being exercised. In this setup, only "agent wait-for bootstrap-complete" will succeed. For vSphere, "agent wait-for install-complete" will fail and is skipped.

andfasano · 2024-02-27T18:11:02Z

/lgtm

rwsu · 2024-02-28T17:56:59Z

/test e2e-agent-compact-ipv4

rwsu · 2024-03-04T15:27:04Z

/test e2e-agent-compact-ipv4

rwsu · 2024-03-04T22:37:43Z

/override e2e-agent-compact-ipv4
Failure is due to an ovn issue that is awaiting fix to be merged downstream: ovn-kubernetes/ovn-kubernetes#4168

openshift-ci · 2024-03-04T22:37:47Z

@rwsu: rwsu unauthorized: /override is restricted to Repo administrators, approvers in top level OWNERS file, and the following github teams:.

Details

In response to this:

/override e2e-agent-compact-ipv4
Failure is due to an ovn issue that is awaiting fix to be merged downstream: ovn-kubernetes/ovn-kubernetes#4168

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

bfournie · 2024-03-04T22:44:37Z

/override e2e-agent-compact-ipv4

openshift-ci · 2024-03-04T22:44:43Z

@bfournie: /override requires failed status contexts, check run or a prowjob name to operate on.
The following unknown contexts/checkruns were given:

e2e-agent-compact-ipv4

Only the following failed contexts/checkruns were expected:

ci/prow/e2e-agent-compact-ipv4
ci/prow/e2e-agent-ha-dualstack
ci/prow/e2e-agent-sno-ipv6
ci/prow/e2e-metal-ipi-bm
ci/prow/e2e-metal-ipi-bm-bond
ci/prow/e2e-metal-ipi-ovn-dualstack
ci/prow/e2e-metal-ipi-ovn-ipv6
ci/prow/e2e-metal-ipi-serial-ipv4
ci/prow/e2e-metal-ipi-serial-ovn-ipv6
ci/prow/e2e-metal-ipi-virtualmedia
ci/prow/images
pull-ci-openshift-metal3-dev-scripts-master-e2e-agent-compact-ipv4
pull-ci-openshift-metal3-dev-scripts-master-e2e-agent-ha-dualstack
pull-ci-openshift-metal3-dev-scripts-master-e2e-agent-sno-ipv6
pull-ci-openshift-metal3-dev-scripts-master-e2e-metal-ipi-bm
pull-ci-openshift-metal3-dev-scripts-master-e2e-metal-ipi-bm-bond
pull-ci-openshift-metal3-dev-scripts-master-e2e-metal-ipi-ovn-dualstack
pull-ci-openshift-metal3-dev-scripts-master-e2e-metal-ipi-ovn-ipv6
pull-ci-openshift-metal3-dev-scripts-master-e2e-metal-ipi-serial-ipv4
pull-ci-openshift-metal3-dev-scripts-master-e2e-metal-ipi-serial-ovn-ipv6
pull-ci-openshift-metal3-dev-scripts-master-e2e-metal-ipi-virtualmedia
pull-ci-openshift-metal3-dev-scripts-master-images
tide

If you are trying to override a checkrun that has a space in it, you must put a double quote on the context.

Details

In response to this:

/override e2e-agent-compact-ipv4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

bfournie · 2024-03-04T22:47:42Z

/override ci/prow/e2e-agent-compact-ipv4

openshift-ci · 2024-03-04T22:47:49Z

@bfournie: Overrode contexts on behalf of bfournie: ci/prow/e2e-agent-compact-ipv4

Details

In response to this:

/override ci/prow/e2e-agent-compact-ipv4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

rwsu · 2024-03-05T16:49:01Z

/retest-required

rwsu · 2024-03-05T20:44:46Z

/retest-required

rwsu · 2024-03-06T22:42:27Z

/retest-required

The job runs the compact_ipv4 scenario for the vsphere platform. The "agent wait-for install-complete" step is skipped because install-complete does not succeed for vsphere using libvirt virtual machines. Depends on: openshift-metal3/dev-scripts#1599 Signed-off-by: Richard Su <rwsu@redhat.com>

rwsu · 2024-03-07T02:24:56Z

/retest-required

rwsu · 2024-03-07T16:25:55Z

/retest-required

rwsu · 2024-03-08T15:20:02Z

/test e2e-metal-ipi-serial-ipv4

openshift-ci · 2024-03-08T21:08:13Z

@rwsu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-metal-ipi-ovn-dualstack	`8738395`	link	false	`/test e2e-metal-ipi-ovn-dualstack`
ci/prow/e2e-metal-ipi-serial-ovn-ipv6	`8738395`	link	false	`/test e2e-metal-ipi-serial-ovn-ipv6`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

rwsu · 2024-03-08T21:57:07Z

/test e2e-metal-ipi-bm

The job runs the compact_ipv4 scenario for the vsphere platform. The "agent wait-for install-complete" step is skipped because install-complete does not succeed for vsphere using libvirt virtual machines. Depends on: openshift-metal3/dev-scripts#1599 Signed-off-by: Richard Su <rwsu@redhat.com>

openshift-ci Bot requested review from lranjbar and pawanpinjarkar November 2, 2023 20:34

openshift-ci Bot requested review from andfasano and bfournie and removed request for lranjbar November 2, 2023 20:35

rwsu force-pushed the AGENT-724 branch from 22638b5 to 06b3ac8 Compare November 8, 2023 23:05

rwsu mentioned this pull request Nov 9, 2023

AGENT-725: Add presubmit job for vsphere openshift/release#45475

Closed

bfournie reviewed Nov 9, 2023

View reviewed changes

rwsu force-pushed the AGENT-724 branch 2 times, most recently from f761d4d to 5abe942 Compare November 9, 2023 20:02

bfournie reviewed Nov 9, 2023

View reviewed changes

Comment thread agent/roles/manifests/tasks/install-config.yml

rwsu force-pushed the AGENT-724 branch from 5abe942 to ee7da43 Compare November 9, 2023 20:52

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 10, 2023

rwsu commented Nov 10, 2023

View reviewed changes

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 9, 2023

rwsu force-pushed the AGENT-724 branch from ee7da43 to 421c421 Compare January 3, 2024 04:10

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 3, 2024

rwsu force-pushed the AGENT-724 branch from 421c421 to 23f2d57 Compare January 30, 2024 22:49

rwsu force-pushed the AGENT-724 branch from 23f2d57 to a3d3d82 Compare February 21, 2024 22:19

andfasano reviewed Feb 22, 2024

View reviewed changes

rwsu force-pushed the AGENT-724 branch from a3d3d82 to 8738395 Compare February 26, 2024 20:51

openshift-ci Bot assigned andfasano Feb 27, 2024

openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Feb 27, 2024

openshift-merge-bot Bot merged commit 532476c into openshift-metal3:master Mar 9, 2024

Conversation

rwsu commented Nov 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rwsu commented Nov 2, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rwsu commented Nov 9, 2023

Uh oh!

Uh oh!

bfournie commented Nov 10, 2023

Uh oh!

openshift-ci Bot commented Nov 10, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andfasano commented Feb 27, 2024

Uh oh!

rwsu commented Feb 28, 2024

Uh oh!

rwsu commented Mar 4, 2024

Uh oh!

rwsu commented Mar 4, 2024

Uh oh!

openshift-ci Bot commented Mar 4, 2024

Uh oh!

bfournie commented Mar 4, 2024

Uh oh!

openshift-ci Bot commented Mar 4, 2024

Uh oh!

bfournie commented Mar 4, 2024

Uh oh!

openshift-ci Bot commented Mar 4, 2024

Uh oh!

rwsu commented Mar 5, 2024

Uh oh!

rwsu commented Mar 5, 2024

Uh oh!

rwsu commented Mar 6, 2024

Uh oh!

rwsu commented Mar 7, 2024

Uh oh!

rwsu commented Mar 7, 2024

Uh oh!

rwsu commented Mar 8, 2024

Uh oh!

openshift-ci Bot commented Mar 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rwsu commented Mar 8, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

rwsu commented Nov 2, 2023 •

edited

Loading

openshift-ci Bot commented Mar 8, 2024 •

edited

Loading