Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run st2-self-check in Github Actions #5609

Merged
merged 26 commits into from
Oct 18, 2023
Merged

Conversation

winem
Copy link
Contributor

@winem winem commented Apr 2, 2022

This PR enables us to run the st2-self-check as part of our CI run in Github Actions as well.

The most crucial part was to handle the environment correctly and to determine the context when running the self-check. A few modifications are needed to adapt to the GHA environment.

Another big change is that the st2-self-check exits with a return code != 0 if any of the tests fail. This is required to ensure that the CI run fails.
I do not see a use-case where this could break anything on the users end but I'll add a note to the changelog to highlight this change as well. I don't think it's a breaking change - but let me know if you disagree.

All tests but 4 (2 remote actions that try to connect to localhost via ssh and 2 HTTP actions) succeed.

I'm open for any input on how to handle the remaining 4 tests.

@armab suggested to have a look at https://github.com/StackStorm/st2packaging-dockerfiles/tree/master/packagingtest and see if we can re-use this here. I'll evaluate it and am open for further ideas.
It's the same for the HTTP actions.

I'll mark this PR as draft until we have a solution for the 4 failing tests but feel free to review the changes I already made and give your feedback!

There may always be tests that may not succeed in GitHub Actions due to special infrastructure, environment or further external dependencies to be satisfied to make them work. So I added some kind of "skip list" or "black list" of tests. These tests will not be executed by GitHub Actions. This list is a space separated list of string configured as environment variable for the self-check job. So you don't have to handle this kind of exceptions in the test files itself and just add them in a single place.

As agreed in a previous TSC meeting the latest Python3.8 release is used.

closes #5496

@pull-request-size pull-request-size bot added the size/L PR that changes 100-499 lines. Requires some effort to review. label Apr 2, 2022
@winem winem marked this pull request as draft April 2, 2022 20:54
@arm4b
Copy link
Member

arm4b commented Apr 3, 2022

The idea with the https://hub.docker.com/r/stackstorm/packagingtest/tags containers is that it comes with an init system and ssh and so you basically use it the same as a VM for testing.
We use them in CI Ansible-st2, Puppet-st2, and st2-packages integration testing.

You can try to use that Docker Image as an environment for installing/running st2.

@winem
Copy link
Contributor Author

winem commented Apr 9, 2022

I found a solution for the ssh issue and these actions work now. Now I'll see if I find any hint why the two HTTP actions fail.

@winem winem force-pushed the 5496-self-check-in-gh-actions branch from f87d6d2 to 16c71b0 Compare September 10, 2022 21:13
@winem
Copy link
Contributor Author

winem commented Sep 10, 2022

The execution of the self-checks works as expected now. I'll add the CHANGELOG-text soon and will also have a look at the failing integration test.

@winem winem marked this pull request as ready for review October 3, 2022 11:09
@winem winem added this to the 3.8.0 milestone Oct 4, 2022
@winem
Copy link
Contributor Author

winem commented Oct 4, 2022

As just agreed in the TSC I'll create issues to run the self-check on other Py versions as well. For now it's Python3.6 that's interesting but Python 3.9 or even 3.10 will become interesting "soon".

There'll be another issue to run the self-check against different OS.

.github/workflows/ci.yaml Show resolved Hide resolved
st2common/bin/st2-self-check Outdated Show resolved Hide resolved
Copy link
Contributor

@nzlosh nzlosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of a generic skip variable, overall this looks good to me.

@winem
Copy link
Contributor Author

winem commented Oct 14, 2023

So I switched to a generic TESTS_TO_SKIP variable that can be used by anyone and added a note to the changelog. So, from my side, this PR is ready to be reviewed again.

@winem winem requested a review from amanda11 October 14, 2023 21:51
Copy link
Member

@arm4b arm4b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works 🎉 Thanks for all the efforts!

A couple of minor comments before merging

CHANGELOG.rst Outdated Show resolved Hide resolved
.github/workflows/ci.yaml Outdated Show resolved Hide resolved
st2common/bin/st2-self-check Outdated Show resolved Hide resolved
@winem winem requested a review from arm4b October 17, 2023 18:34
Copy link
Member

@arm4b arm4b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @winem!
Thanks!

@arm4b arm4b merged commit 4d58a17 into master Oct 18, 2023
38 checks passed
@arm4b arm4b deleted the 5496-self-check-in-gh-actions branch October 18, 2023 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement size/L PR that changes 100-499 lines. Requires some effort to review.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

CI should run st2-self-check for every PR
5 participants