Service accounts should be restored prior to workload controllers

[jhorwit2]

$ ark version
Version: v0.6.0
Git commit: v0.6.0
Git tree state: clean

I ran a backup restore earlier and noticed that service accounts were not created before the workload controllers which causes the restore process to take longer and at first it makes it seem like the restore process had an issue. It'd be nice if there was some ordering of restores to avoid unnecessary events and further delay.

$ ark restore create monitoring
Restore "monitoring-20171209222726" created successfully.

$ kba -n monitoring get ds
NAME         DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR                         AGE
fluentd      0         0         0         0            0           <none>                                9s
prometheus   0         0         0         0            0           node-role.kubernetes.io/prometheus=   9s

$ kba -n monitoring get sa
NAME      SECRETS   AGE
default   1         47s

$ kba -n monitoring get sa
NAME         SECRETS   AGE
default      1         1m
fluentd      3         33s
prometheus   3         33s

$ kba -n monitoring get ds
NAME         DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR                         AGE
fluentd      10        10        10        10           10          <none>                                1m
prometheus   0         0         0         0            0           node-role.kubernetes.io/prometheus=   1m

Events:
  Type     Reason        Age               From        Message
  ----     ------        ----              ----        -------
  Warning  FailedCreate  3m (x14 over 4m)  daemon-set  Error creating: pods "prometheus-" is forbidden: service account monitoring/prometheus was not found, retry after the service account is created

[ncdc]

Recording our conversation from slack: it is possible to do this, but it's not a default setting. You can edit the config's resourcePriorities. The default, if unset, is namespaces, persistentvolumes, persistentvolumeclaims, secrets, configmaps. You would need to add serviceaccounts to this list. Anything not in the list is restored after all of these.

We will also do a PR to change the default.