Make systemd service dependencies considerably more strict

[sdodson]

Before/After enforces ordering and ordering only.
Wants triggers the named dependency to start but will not prevent the service from starting in the event that the dependency has failed. I've only used Requires in the case of the openshift-node->docker dependency because the other services could potentially be located on other hosts or may not even be used/installed like etcd so I believe Wants is more appropriate for those.

[sdodson]

Similar changes for openshift-sdn in openshift/openshift-sdn#44

[sdodson]

@brenton mind taking a look and giving this a merge?

[sdodson]

@sosiouxme mind taking a look? This follows similar changes made in openshift-sdn. We're perhaps overly descriptive, including services that aren't directly connected to one another, but I was erring on the side of each unit file accurately describing where it falls into the complete system.

[brenton]

Pardon my systemd unit file stupidity. What does this do if the etcd.service doesn't exist? (like in the case of use the embedded one)

[sdodson]

It will log that they're not found and move on unless it's a Requires, in which case it would prevent the service from starting.

[sdodson]

This does mean that if etcd were installed, but we're configured to use the embedded etcd, it would attempt to start etcd and we'd run into problems binding to the port assuming it's not reconfigured to use another port. But, uhhh, don't install etcd if you're not going to use it?

[sosiouxme]

@brenton I'm only moderately up on unit file semantics but with systemd, load order and dependency are orthogonal concepts... See Requires/Wants vs Before/After at http://www.freedesktop.org/software/systemd/man/systemd.unit.html

I don't feel great about "don't install it if you don't want it" because it's nice to be able to just create an image with everything on it, then only activate the parts you want on a particular host. I guess you could just nuke stuff you don't want, but I wonder if there's a better way to indicate "Service A can do this or rely on Service B, but we don't want to do both"... going to think about it for a minute.

[sosiouxme]

Discussed, to summarize:

• If we just do After= everywhere, then assume the user (or installer) will systemctl enable whatever they actually want running, everything works out at boot time (as services pretty much are started by the same boot target).
• The concern then is only when people are manually stopping/starting services, trying to make sure that if something is enabled and supposed to run before something else, it actually does, but not if it's not enabled. The [Install] section WantedBy seems to be ideal for this purpose if it works, but for some reason this is not recommended... see if/how it works.
• Since I really don't think we generally want e.g. the master to start etcd unless etcd is enabled (least surprise principle), what we could do if the above is inadequate is have the install add e.g. Wants=etcd.service into /etc/systemd/system/openshift-master.d/etcd.conf (see Example 2. Overriding vendor settings) when it knows the master is depending on remote etcd. The top concern with this would be that the average sysadmin might not know to look for these overrides.

[sdodson]

I've confirmed the second bullet point.

First, lets assume openshift*, docker, openvswitch, and etcd are installed on the host. All services are disabled and stopped. We're ignoring SDN related stack for now.

We want this to be a node and a node only, so we enable and start openshift-node. Assuming it's configured properly for a master everything is fine, no additional services are started.

Next, we want this to be an all in one with the master on the same host. We re-configure the node to use the local master (outside of the scope of the systemd service definitions) and we enable but do not start openshift-master service. Now whenever we start openshift-node it will start openshift-master due to this symlink created when systemctl enable openshift-master

[root@ose3-master ~]# systemctl enable openshift-master
ln -s '/usr/lib/systemd/system/openshift-master.service' '/etc/systemd/system/multi-user.target.wants/openshift-master.service'
ln -s '/usr/lib/systemd/system/openshift-master.service' '/etc/systemd/system/openshift-node.service.wants/openshift-master.service'

This happens because we've added WantedBy=openshift-node.service to the [Install] section of openshift-master.service

[Install]
WantedBy=multi-user.target
WantedBy=openshift-node.service

I believe bullet point two plus instructions or installer automation enabling etcd when it's intended to be used is sufficient. We'll still have After=etcd.service in the master to ensure ordering.

[sosiouxme]

+1 :)

[sdodson]

I think this should be ready for merge. @sosiouxme mind pressing the big button?

[sosiouxme]

[merge]

[openshift-bot]

continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/merge_pull_requests_openshift3/1492/) (Image: devenv-fedora_1222)

[sosiouxme]

https://ci.openshift.redhat.com/jenkins/job/merge_pull_requests_openshift3/1457/ seems spurious so [merge] again

[sdodson]

[merge] again, i think I read somewhere that subsequent merge requests don't require ACL checks.

[sosiouxme]

Alas poor https://ci.openshift.redhat.com/jenkins/job/merge_pull_requests_openshift3/1458/

Let's try another [merge] and hope for a better day.

[openshift-bot]

Evaluated for origin up to 32a35f5