Deletes cluster when upgrade fails

[cehoffman]

There is an issue I opened in etcd etcd-io/etcd#8268 that highlighted what I think is a bug in expected behavior for the operator. When upgrading a cluster in this case, the operator deletes the entire cluster instead of reverting the cluster to the old etcd image.

It would seem preferable to me that the cluster enters a degraded state or reverts fully to the previous version instead of straight deletion. The reason for deletion in the logs is that no backup configuration exists in the spec.

[hongchaodeng]

So you mean when upgrade halted in the middle, revert to previous version?

To extend this topic, currently you can backup before upgrade and if anything goes wrong just restore fro it.

[hongchaodeng]

We can possibly add a timeout in upgrade.

[cehoffman]

The intention is that when upgrading, if upgraded members do not actually become part of the cluster the upgrade should be stopped or rolled back.

[xiang90]

The intention is that when upgrading, if upgraded members do not actually become part of the cluster the upgrade should be stopped or rolled back.

Right. However, it is unplanned right now. Can you open a feature request for the upgrade rollback feature

[cehoffman]

Done