Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Add tags to TF workflow, allow more [citest bad] formats #224

Merged
merged 1 commit into from
Aug 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions .github/workflows/tft.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ jobs:
PR_NO: ${{ github.event.issue.number }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Checkout PR
uses: actions/checkout@v4
with:
ref: ${{ steps.head_sha.outputs.head_sha }}

- name: Get memory
id: memory
run: |
Expand Down Expand Up @@ -113,17 +118,17 @@ jobs:
ARTIFACTS_DIR_NAME: "tf_${{ github.event.repository.name }}-${{ github.event.issue.number }}_\
${{ matrix.platform }}-${{ matrix.ansible_version }}_\
${{ needs.prepare_vars.outputs.datetime }}/artifacts"
ARTIFACT_TARGET_DIR: /srv/pub/alt/linuxsystemroles/logs
ARTIFACT_TARGET_DIR: /srv/pub/alt/${{ vars.LINUXSYSTEMROLES_USER }}/logs
steps:
- name: Set variables with DATETIME and artifact location
id: set_vars
run: |
printf -v DATETIME '%(%Y%m%d-%H%M%S)T' -1
ARTIFACTS_DIR_NAME="tf_${{ github.event.repository.name }}-${{ github.event.issue.number }}_\
${{ matrix.platform }}-${{ matrix.ansible_version }}_$DATETIME/artifacts"
ARTIFACTS_TARGET_DIR=/srv/pub/alt/linuxsystemroles/logs
ARTIFACTS_TARGET_DIR=/srv/pub/alt/${{ vars.LINUXSYSTEMROLES_USER }}/logs
ARTIFACTS_DIR=$ARTIFACTS_TARGET_DIR/$ARTIFACTS_DIR_NAME
ARTIFACTS_URL=https://dl.fedoraproject.org/pub/alt/linuxsystemroles/logs/$ARTIFACTS_DIR_NAME
ARTIFACTS_URL=https://dl.fedoraproject.org/pub/alt/${{ vars.LINUXSYSTEMROLES_USER }}/logs/$ARTIFACTS_DIR_NAME
echo "DATETIME=$DATETIME" >> $GITHUB_OUTPUT
echo "ARTIFACTS_DIR=$ARTIFACTS_DIR" >> $GITHUB_OUTPUT
echo "ARTIFACTS_URL=$ARTIFACTS_URL" >> $GITHUB_OUTPUT
Expand Down Expand Up @@ -155,17 +160,18 @@ jobs:
git_url: https://github.com/linux-system-roles/tft-tests
git_ref: main
pipeline_settings: '{ "type": "tmt-multihost" }'
environment_settings: '{ "provisioning": { "tags": { "BusinessUnit": "system_roles" } } }'
# Keeping ARTIFACTS_URL at the bottom makes the link in logs clickable
variables: "ANSIBLE_VER=${{ matrix.ansible_version }};\
REPO_NAME=${{ github.event.repository.name }};\
GITHUB_ORG=${{ github.repository_owner }};\
GITHUB_ORG=linux-system-roles;\
PR_NUM=${{ github.event.issue.number }};\
ARTIFACTS_DIR=${{ steps.set_vars.outputs.ARTIFACTS_DIR }};\
ARTIFACTS_URL=${{ steps.set_vars.outputs.ARTIFACTS_URL }};\
TEST_LOCAL_CHANGES=false"
TEST_LOCAL_CHANGES=false;\
LINUXSYSTEMROLES_USER=${{ vars.LINUXSYSTEMROLES_USER }};\
ARTIFACTS_URL=${{ steps.set_vars.outputs.ARTIFACTS_URL }}"
# Note that LINUXSYSTEMROLES_SSH_KEY must be single-line, TF doesn't read multi-line variables fine.
secrets: "LINUXSYSTEMROLES_USER=${{ secrets.LINUXSYSTEMROLES_USER }};\
LINUXSYSTEMROLES_DOMAIN=${{ secrets.LINUXSYSTEMROLES_DOMAIN }};\
secrets: "LINUXSYSTEMROLES_DOMAIN=${{ secrets.LINUXSYSTEMROLES_DOMAIN }};\
LINUXSYSTEMROLES_SSH_KEY=${{ secrets.LINUXSYSTEMROLES_SSH_KEY }}"
compose: ${{ matrix.platform }}
# There are two blockers for using public ranch:
Expand All @@ -175,6 +181,7 @@ jobs:
api_key: ${{ secrets.TF_API_KEY_RH }}
update_pull_request_status: false
tmt_hardware: '{ "memory": ">= ${{ needs.prepare_vars.outputs.memory }} MB" }'
tmt_plan_filter: "tag:general,kernel_settings"

- name: Set final commit status
uses: myrotvorets/set-commit-status-action@master
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tft_citest_bad.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
citest_bad_rerun:
if: |
github.event.issue.pull_request
&& github.event.comment.body == '[citest_bad]'
&& contains(fromJson('["[citest_bad]", "[citest-bad]", "[citest bad]"]'), github.event.comment.body)
&& contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR", "CONTRIBUTOR"]'), github.event.comment.author_association)
permissions:
actions: write # for re-running failed jobs: https://docs.github.com/en/rest/actions/workflow-runs?apiVersion=2022-11-28#re-run-a-job-from-a-workflow-run
Expand Down
115 changes: 115 additions & 0 deletions contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,118 @@ are likely to be suitable for new contributors!

**Code** is managed on [Github](https://github.com/linux-system-roles/kernel_settings), using
[Pull Requests](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests).

## Python Code

The Python code needs to be **compatible with the Python versions supported by
the role platform**.

For example, see [meta](https://github.com/linux-system-roles/kernel_settings/blob/main/meta/main.yml)
for the platforms supported by the role.

If the role provides Ansible modules (code in `library/` or `module_utils/`) -
these run on the *managed* node, and typically[1] use the default system python:

* EL6 - python 2.6
* EL7 - python 2.7 or python 3.6 in some cases
* EL8 - python 3.6
* EL9 - python 3.9

If the role provides some other sort of Ansible plugin such as a filter, test,
etc. - these run on the *control* node and typically use whatever version of
python that Ansible uses, which in many cases is *not* the system python, and
may be a modularity release such as python311.

In general, it is a good idea to ensure the role python code works on all
versions of python supported by `tox-lsr` from py36 on, and on py27 if the role
supports EL7, and on py26 if the role supports EL6.[1]

[1] Advanced users may set
[ansible_python_interpreter](https://docs.ansible.com/ansible/latest/reference_appendices/special_variables.html#term-ansible_python_interpreter)
to use a non-system python on the managed node, so it is a good idea to ensure
your code has broad python version compatibility, and do not assume your code
will only ever be run with the default system python.

## Testing kernel_settings modules

It is recommended to use `tox` to set up your `virtualenv` for
development/testing purposes:

```bash
dnf/yum install python-tox
tox -e py38
```

You can also use the virtualenv created by `tox` just like any
other virtualenv created by `python-virtualenv`:

```bash
. .tox/env-py38/bin/activate
python
>>> import package.that.only.exists.in.venv
```

The unit tests and other tests are run by default when you use `tox` by itself
or `tox -e py38` for a specific python versioned environment. Note that other
operating system packages may be required to be installed in order for `tox`
to use `pip` to install python dependencies e.g. for python packages which
have native components.

I would also strongly encourage you to use an IDE for development. For example,
Visual Studio code python extension auto-discovers tests and allows you to
run and debug unit tests. However, you may need to create a `.env` file like
this, in order for code navigation, auto-completion, and test discovery to
work correctly:

```bash
PYTHONPATH=/full/path/to/tuned:/full/path/to/linux-system-roles/kernel_settings/library
```

### Testing the module

[Ansible Module Development Guide](https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_general.html)

Using a tox python virtualenv from the `kernel_settings` directory:

```bash
. .tox/env-py38/bin/activate
TESTING=true [TEST_PROFILE=kernel_settings] python \
library/kernel_settings.py args.json
```

looks for test profiles under `tests/tuned/etc/tuned`

to run the code in the debugger:

```bash
TESTING=true [TEST_PROFILE=kernel_settings] python -mpdb \
library/kernel_settings.py args.json
```

Where `args.json` looks like this:

```json
{
"ANSIBLE_MODULE_ARGS": {
"name": "kernel_settings",
"sysctl": [
{"name": "fs.inotify.max_user_watches", "value": 524288},
{"name": "kernel.threads-max", "value": 30001}
],
"sysfs": [
{"name": "/sys/kernel/kexec_crash_size", "value": 337641472}
],
"bootloader": [
{"name": "cmdline", "value": [
{"name": "mitigations", "value": "on"},
{"name": "another"}
]
}
],
"selinux": [
{"name": "avc_cache_threshold", "value": 512}
],
"purge": false
}
}
```
Loading