ci: add support for /retest command

[jpsim]

Commenting /retest on a PR will re-run failed jobs in its GitHub Actions workflows.

Copied from https://github.com/containernetworking/plugins.

[jpsim]

The /retest command will only work once this PR is merged, so I can't test it here yet.

[jpsim]

Right now, this will re-run all CI jobs, including ones that are already succeeding. I might look into making it just re-run failing jobs at a later time.

Update: now only re-runs failed jobs

[jpsim]

In my testing, it takes 20 to 30 seconds for the command to take effect: https://github.com/jpsim/ZenTuner/actions/workflows/commands.yml

[keith]

there is some custom stuff on envoy that we could reuse here probably

[jpsim]

there is some custom stuff on envoy that we could reuse here probably

i looked at that but Envoy’s CI is all Azure Pipelines so it doesn’t really carry over.

[mattklein123]

cc @itayd who helps us with RepoKitteh in the main repo and is working on "something better" which we can hopefully use here also when it is ready.

[itayd]

don't we already have retest working here for azp?

[jpsim]

Envoy Mobile uses GitHub Actions for all of its CI, there are no azp jobs. There's also one CircleCI job that deploys docs when new versions are pushed.

[Augustyniak]

Speaking from the perspective of a person who did a few contributions in EM repo without being a maintainer I think that having a /retest command is a quality of life improvement - especially if we find a way to eventually make it smarter so that it does not rerun green jobs (that would've been pretty awesome).

Pretty often I found myself pushing multiple empty commits to a branch just to get rid jobs that decided to go red randomly. Having /retest would've allowed me to have to do less of a git work.

[goaway]

Now that GitHub Actions directly supports rerunning failed jobs only, I think that's almost always preferable to rerunning all of CI. We have finite resources, and CI jobs do get backed up. If this could be improved to rerun failed jobs only, I feel it would be a lot more compelling. That said, for the occasions where rerunning everything really is what you want (post-CI outage?), I don't see the harm in having this.

[jpsim]

Thanks for the feedback everyone. After spending some time looking at the API responses, and looking at GitHub's REST API documentation, I found an endpoint to rerun failed jobs from a workflow run.

Unlike rerun_url, this endpoint isn't in the run API response, but we can construct it manually easily.

Documentation for it is here: https://docs.github.com/en/rest/actions/workflow-runs#re-run-failed-jobs-from-a-workflow-run

I've tested it on a separate repo and it worked as expected: jpsim/ZenTuner#47

[jpsim]

@goaway @Augustyniak did my latest updates address your concerns about only re-running failing jobs?

[goaway]

Thanks @jpsim! This is fantastic, and should be a huge improvement over needing to navigate the UI to restart each failing individually - in addition to solving the permissions issue for external contributors.