WIP: Static ip

[derekhiggins]

Cluster eventually comes up and completes

Continuing the work started here
#1297

TODO:

• bootstrap consistently times out when using static IPs, cluster eventually comes up after the installer has died
• need to make dhcp config configurable
• test ipv6

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please ask for approval from derekhiggins after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hardys]

need to make dhcp config configurable

FYI I started looking at that via metal3-io/metal3-dev-env#902 (now merged) - with that we can do something like this:

diff --git a/vm_setup_vars.yml b/vm_setup_vars.yml
index a345ce5..f3b60ed 100644
--- a/vm_setup_vars.yml
+++ b/vm_setup_vars.yml
@@ -61,13 +61,8 @@ external_network:
     netmask_v4: "{{ baremetal_network_cidr_v4|ipaddr('netmask') }}"
     address_v6: "{{ baremetal_network_cidr_v6|nthhost(1)|default('', true) }}"
     prefix_v6: "{{ baremetal_network_cidr_v6|ipaddr('prefix') }}"
-    dhcp_enabled: "{{ virsh_dhcp_enabled|bool }}"
-    dhcp_range_v4:
-      - "{{ baremetal_network_cidr_v4|nthhost(20) }}"
-      - "{{ baremetal_network_cidr_v4|nthhost(60) }}"
-    dhcp_range_v6:
-      - "{{ baremetal_network_cidr_v6|nthhost(20) }}"
-      - "{{ baremetal_network_cidr_v6|nthhost(60) }}"
+    dhcp_range_v4: "{{ [ baremetal_network_cidr_v4|nthhost(20), baremetal_network_cidr_v4|nthhost(60) ] if lookup('env', 'STATIC_IPS') == '' else [] }}"
+    dhcp_range_v6: "{{ [ baremetal_network_cidr_v6|nthhost(20), baremetal_network_cidr_v6|nthhost(60) ] if lookup('env', 'STATIC_IPS') == '' else [] }}"
     lease_expiry: "{{ lookup('env', 'DHCP_LEASE_EXPIRY') }}"
     nat_port_range:
       - 1024

[cybertron]

Having a shortcut to use static ips is useful, but I'm more interested in an interface to provide arbitrary network configs for the nodes so we can test things like bridge and bond configs. This is fairly important so we can duplicate (as close as possible, anyway) qe and customer network configurations.

Since networkConfigs are all multi-line, I don't think we want to try to pass them as env vars. I think we need an env var that points to a file or files containing the relevant networkConfigs. My first thought is to have the env var point to a directory (like ASSETS_EXTRA_FOLDER) and inside that there would be specific files - master-0.yaml, master-1.yaml, etc. containing the networkConfig for the respective node, as well as a vm_setup_extrahosts.yml for defining necessary DNS records.

None of this eliminates the need for a static ip shortcut option, but I'm wondering if it makes more sense to implement the arbitrary interface and then do the static ip config using that instead of as a one-off like it is now.

[cybertron]

I tried this locally and for some reason my workers are stuck on the agent image. I see a few errors in the logs, but it mostly looks like optional stuff and the agent run seems to complete so I'm not sure what's going on there. It's not specific to this patch because when I manually configured static IPs it did the exact same thing.

[cybertron]

To wrap up my previous comment, I believe there was an actual bug with networkConfig causing the failure. That's fixed now.

I pushed #1418 as an example of what I was thinking for a generic networkConfig interface. It still needs something to add DNS records for static IPs, but with a little vm_setup_vars hacking it worked well. There should be some way to get the dns_extrahosts from the same directory.

[openshift-merge-robot]

@derekhiggins: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci]

@derekhiggins: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-metal-ipi-serial-ovn-ipv6	e8fb543	link	true	/test e2e-metal-ipi-serial-ovn-ipv6
ci/prow/e2e-metal-ipi-ovn-dualstack	e8fb543	link	false	/test e2e-metal-ipi-ovn-dualstack
ci/prow/e2e-metal-ipi-proxy-ipv6	e8fb543	link	true	/test e2e-metal-ipi-proxy-ipv6
ci/prow/e2e-metal-ipi-proxy-ipv4	e8fb543	link	true	/test e2e-metal-ipi-proxy-ipv4
ci/prow/e2e-metal-ipi-ovn-ipv6	e8fb543	link	true	/test e2e-metal-ipi-ovn-ipv6
ci/prow/e2e-agent-compact	e8fb543	link	true	/test e2e-agent-compact
ci/prow/e2e-metal-ipi-serial-ipv4	e8fb543	link	true	/test e2e-metal-ipi-serial-ipv4
ci/prow/e2e-metal-ipi	e8fb543	link	true	/test e2e-metal-ipi
ci/prow/e2e-metal-ipi-bm	e8fb543	link	true	/test e2e-metal-ipi-bm

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[derekhiggins]

Closing as this was handled in #1418