Possibility for serialization within parallel processed groups

[tuckson]

Proposal: Add possibility for serialization within parallel processed groups

Author: Your Name <@tuckson> IRC: handle (if different)

Date: 2019-07-12

• Status: New
• Proposal type: (core design)
• Targeted release: <future release #, ex. 2.2>
• Associated PR: <link to GH PR in ansible/proposals if PR was submitted>
• Estimated time to implement: <X days, weeks, etc.>

Motivation

In ansible we can control how many servers are processed at the same time via the 'serial' parameter. However in a large, loadbalanced environment there is another architectural thing we need to worry about. These are the vips inside the loadbalancers. In short: many of may servers are grouped in pools in a loadbalancer. And while I do not want to process more then 2 memmbers of such a pool at the same time to avoind performance degradation for my customers, it makes perfectly sense to process servers in other pools at the same time (but also not more than 2 servers of such a pool). At this moment I see no way of achieving that inside ansible. Hence this feature request.

Problems

What problems exist that this proposal will solve?

• unability to process parallel updates without bringing vips down
  At current we have to either use multiple playbooks for the same task in different microservices or just accept processing is taking a long time. The requested feature would be a great help in reducing time to process certain (larger) automated processes while still keeping things safe for our customers.

Solution proposal

Suppose the top part of the playbook:

- name: Wedosomething
   hosts: allservers
   vars:
       srvdowntime: 120
   gather_facts: yes
   become: true
   serial: 2
   groups-parallel: yes

In the inventory:

[allservers:children]
Group1
Group2
Group3

[Group1]
Server[1:5]

[Group2]
Server[6:10]

[Group3]
Server[11:15]

If groups-parallel is set to 'yes' then 'serial: 2' scopes to childgroup level. In that case ansible starts and processes these servers at the same time:
Server1, Server2,Server6,Server7.Server11,Server12
(And it goes on per group until al servers are finished).

If groups-parallel is set to 'no' or is it (default) absent, than 'Serial: 2' scopes for to all hosts and ansible starts with processing
Server1, Server2
at the same time (and proceeds after that).

Without Serial groups-parallel is of no use. When you put a groupname without [:children] after 'hosts: ' neither. And the same goes for 'hosts: all'.

Dependencies (optional)

Explain any dependencies. This section is optional but could be helpful.

• Dependency Docker Modules #1
• Dependency Adding Docker module proposals #2

Testing (optional)

Does / should this require testing, and if so, how? Describe here. This section is optional but could be helpful.
Absolutely. Since this will require you to change the parts that process the inventory I suppose some serious testing would be nice.

Documentation (optional)

Does / should this require documentation? If so, describe here. This section is optional but could be helpful.
Yes, The documentation should describe the use and limits of the new parameter.

Anything else?

Feel free to ask for clarification

[bcoca]

There are several things assumed/required above that are not part of how groups work:

• Hosts are only part of one group on definition: They are ALWAYS part of at least 2 groups.
• Where a host is defined is discoverable: A host can be defined multiple times, we only record the properties of each definition and aggregate them, in the end the host will be part of child/parents w/o distinction on how they were defined.
• parent child relationships are traversed at play iteration: no, groups are only used to narrow down host selection, they are in the end just a 'label/property' of the host and a convenient way to assign variables to multiple hosts.

So to implement this proposal we would have to radically change how groups are handled internally.