Fix node scaleup plays

[michaelgugino]

Currently, users have no way to run preqrequisites.yml
on just newly added nodes during scaleup.

This commit ensures only the new nodes are changed during
scaleup as well as ensure prerequisites are run.

[michaelgugino]

Latest patch set yields really good results. I deployed 1 master + 1 node initially; Then I ran scaleup against two new does. Here are the task counts on the scaleup play:

node1 : ok=3    changed=0    unreachable=0    failed=0   
node2 : ok=163  changed=55   unreachable=0    failed=0   
node3 : ok=168  changed=58   unreachable=0    failed=0   
mast1 : ok=31   changed=0    unreachable=0    failed=0 

[mtnbikenc]

I've not run a test build but the changes look good.
/lgtm

[vrutkovs]

This also needs to include os_firewall role, as currently node gets setup correctly, but node registration hangs up if firewall is enabled

[fraenkel]

I am a bit concerned that node scaleup calls prerequisites. To my knowledge, one must run prereqs first before running a playbook since it doesn't happen with deploy-cluster or master scaleup.

[michaelgugino]

I am a bit concerned that node scaleup calls prerequisites. To my knowledge, one must run prereqs first before running a playbook since it doesn't happen with deploy-cluster or master scaleup.

I thought about this a good deal. The issue is, prerequisites.yml runs against several host groups; having the user call prerequisites.yml without a properly constructed inventory could lead to undesired changes. We would need to create a scaleup_prerequisites.yml to check that new_nodes is defined in inventory and only execute against those.

[michaelgugino]

This also needs to include os_firewall role, as currently node gets setup correctly, but node registration hangs up if firewall is enabled

os_firewall role is called during prerequisites.yml

[fraenkel]

If this is how it works for node scaleup, how does master scale up work since it too is a node?
And for me, I am running containerized so I have to guarantee all of our instances run prerequisites anyhow.

For now we have been creating the inventory the way it needs to look and then run prereq with a --limit to target the instances since we have seen all be run instead of what we care about.

[michaelgugino]

If this is how it works for node scaleup, how does master scale up work since it too is a node?
And for me, I am running containerized so I have to guarantee all of our instances run prerequisites anyhow.

It appears that we can implement similar logic to this in the scaleup play for masters. At that time, any additional masters and nodes would be configured; If you're adding both nodes and masters, just run the master scaleup play. If you're adding just nodes, run the node scaleup play.

For now we have been creating the inventory the way it needs to look and then run prereq with a --limit to target the instances since we have seen all be run instead of what we care about.

There are many ways to accomplish things in ansible. Advanced users could easily construct an inventory on the fly and use the --limit switch to achieve the desired behavior. Unfortunately, not everyone that interacts with openshift-ansible is an advanced ansible user, so we have to make our plays as accommodating as possible for a wide variety of use cases.

I am open to suggestion on this topic, but we need to try to keep the work flow as simple as possible.

[michaelgugino]

/test install

[openshift-merge-robot]

/test all [submit-queue is verifying that this PR is safe to merge]

[openshift-ci-robot]

@michaelgugino: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/openshift-jenkins/tox	6646d02	link	/test tox
ci/openshift-jenkins/install	6646d02	link	/test install
ci/openshift-jenkins/extended_conformance_install_crio	6646d02	link	/test crio

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.