what happens on failures?

[tcurdt]

What happens when something fails after a deploy?
Is there any way to auto roll back?

[nlewo]

@tcurdt Currently, comin doesn't have anything to detect a failure after a deployment, so it can't do anything on failure.

Do you have something in mind?

Maybe comin could watch a liveness probe in order to be able to rollback. However, i don't know if it should watch for some prometheus metrics, a systemd unit state or executing a script after the deployment to ensure the machine still works.

(When i'm not sure about a deployment, i currently use a testing branch in order to be able to reboot the server if it is no longer responding.)

[tcurdt]

I guess the main problem is defining what is considered a system that is alive and well.
As you already hinted, things that come to mind are:

• http ping
• prometheus metric
• systemd unit state
• script exit code

All of them would need some kind of grace period after the deploy. If any of them failed, rollback to a working state.
But what happens on the next pull? I guess it would need to keep the SHAs that were tried (somewhere).

[tcurdt]

Another somewhat related thought:

Is there a way to check the current SHA of the system yet?
I see it is mentioned for the prometheus metrics.

[tcurdt]

Ah, seems like the status page provides it

curl 127.0.0.1:4242/status