Skip to content

test: run unit-tests on forks #29979

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
iamsobanjaved merged 2 commits into from
Mar 31, 2022
Merged

test: run unit-tests on forks #29979

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6ccadbb
test: run unit-tests on forks
iamsobanjaved Feb 16, 2022
aa742f0
Merge branch 'master' into iamsobanjaved/unit-tests-for-forks
iamsobanjaved Mar 31, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/actions/unit-tests/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: 'Run unit tests'
description: 'shared steps to run unit tests on both Github hosted and self hosted runners.'
runs:
using: "composite"
steps:
- name: set settings path
shell: bash
run: |
echo "settings_path=$(python scripts/unit_test_shards_parser.py --shard-name=${{ matrix.shard_name }} --output settings )" >> $GITHUB_ENV

- name: get unit tests for shard
shell: bash
run: |
echo "unit_test_paths=$(python scripts/unit_test_shards_parser.py --shard-name=${{ matrix.shard_name }} )" >> $GITHUB_ENV

- name: run tests
shell: bash
run: |
python -Wd -m pytest -p no:randomly --ds=${{ env.settings_path }} ${{ env.unit_test_paths }}

- name: rename warnings json file
if: success()
shell: bash
run: |
cd test_root/log
mv pytest_warnings.json pytest_warnings_${{ matrix.shard_name }}.json

- name: save pytest warnings json file
if: success()
uses: actions/upload-artifact@v2
with:
name: pytest-warnings-json
path: |
test_root/log/pytest_warnings*.json
49 changes: 49 additions & 0 deletions .github/actions/verify-tests-count/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
name: 'Verify unit tests count'
description: 'shared steps to verify unit tests count on both Github hosted and self hosted runners.'
runs:
using: "composite"
steps:
- name: collect tests from all modules
shell: bash
run: |
echo "root_cms_unit_tests_count=$(pytest --collect-only --ds=cms.envs.test cms/ -q | head -n -2 | wc -l)" >> $GITHUB_ENV
echo "root_lms_unit_tests_count=$(pytest --collect-only --ds=lms.envs.test lms/ openedx/ common/djangoapps/ common/lib/ -q | head -n -2 | wc -l)" >> $GITHUB_ENV

- name: get GHA unit test paths
shell: bash
run: |
echo "cms_unit_test_paths=$(python scripts/gha_unit_tests_collector.py --cms-only)" >> $GITHUB_ENV
echo "lms_unit_test_paths=$(python scripts/gha_unit_tests_collector.py --lms-only)" >> $GITHUB_ENV


- name: collect tests from GHA unit test shards
shell: bash
run: |
echo "cms_unit_tests_count=$(pytest --collect-only --ds=cms.envs.test ${{ env.cms_unit_test_paths }} -q | head -n -2 | wc -l)" >> $GITHUB_ENV
echo "lms_unit_tests_count=$(pytest --collect-only --ds=lms.envs.test ${{ env.lms_unit_test_paths }} -q | head -n -2 | wc -l)" >> $GITHUB_ENV


- name: add unit tests count
shell: bash
run: |
echo "root_all_unit_tests_count=$((${{ env.root_cms_unit_tests_count }}+${{ env.root_lms_unit_tests_count }}))" >> $GITHUB_ENV
echo "shards_all_unit_tests_count=$((${{ env.cms_unit_tests_count }}+${{ env.lms_unit_tests_count }}))" >> $GITHUB_ENV

- name: print unit tests count
shell: bash
run: |
echo CMS unit tests from root: ${{ env.root_cms_unit_tests_count }}
echo LMS unit tests from root: ${{ env.root_lms_unit_tests_count }}
echo CMS unit tests from shards: ${{ env.cms_unit_tests_count }}
echo LMS unit tests from shards: ${{ env.lms_unit_tests_count }}
echo All root unit tests count: ${{ env.root_all_unit_tests_count }}
echo All shards unit tests count: ${{ env.shards_all_unit_tests_count }}

- name: fail the check
shell: bash
if: ${{ env.root_all_unit_tests_count != env.shards_all_unit_tests_count }}
run: |
echo "::error title='Unit test modules in unit-test-shards.json (unit-tests.yml workflow) are outdated'::unit tests running in unit-tests
workflow don't match the count for unit tests for entire edx-platform suite, please update the unit-test-shards.json under .github/workflows
to add any missing apps and match the count. for more details please take a look at scripts/gha-shards-readme.md"
exit 1
113 changes: 113 additions & 0 deletions .github/workflows/unit-tests-gh-hosted.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
name: unit-tests-gh-hosted

on:
pull_request:
push:
branches:
- master
- open-release/lilac.master

jobs:
run-test:
if: github.repository != 'openedx/edx-platform' && github.repository != 'edx/edx-platform-private'
runs-on: ubuntu-20.04
strategy:
fail-fast: false
matrix:
python-version: [ '3.8' ]
django-version: [ "3.2" ]
shard_name: [
"lms-1",
"lms-2",
"lms-3",
"lms-4",
"lms-5",
"lms-6",
"openedx-1",
"openedx-2",
"openedx-3",
"openedx-4",
"cms-1",
"cms-2",
"common-1",
"common-2",
"common-3",
]
name: gh-hosted-python-${{ matrix.python-version }},django-${{ matrix.django-version }},${{ matrix.shard_name }}
steps:
- uses: actions/checkout@v2

- name: Install Required System Packages
run: sudo apt-get update && sudo apt-get install libxmlsec1-dev lynx

- name: Start MongoDB
uses: supercharge/mongodb-github-action@1.7.0
with:
mongodb-version: 4.4

- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Get pip cache dir
id: pip-cache-dir
run: |
echo "::set-output name=dir::$(pip cache dir)"

- name: Cache pip dependencies
id: cache-dependencies
uses: actions/cache@v2
with:
path: ${{ steps.pip-cache-dir.outputs.dir }}
key: ${{ runner.os }}-pip-${{ hashFiles('requirements/edx/development.txt') }}
restore-keys: ${{ runner.os }}-pip-

- name: Install Required Python Dependencies
run: |
pip install -r requirements/pip.txt
pip install -r requirements/edx/development.txt --src ${{ runner.temp }}
pip install "django~=${{ matrix.django-version }}.0"

- name: Setup and run tests
uses: ./.github/actions/unit-tests

collect-and-verify:
if: github.repository != 'openedx/edx-platform' && github.repository != 'edx/edx-platform-private'
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [ '3.8' ]
django-version: [ "3.2" ]
steps:
- uses: actions/checkout@v2

- name: Install Required System Packages
run: sudo apt-get update && sudo apt-get install libxmlsec1-dev

- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Get pip cache dir
id: pip-cache-dir
run: |
echo "::set-output name=dir::$(pip cache dir)"

- name: Cache pip dependencies
id: cache-dependencies
uses: actions/cache@v2
with:
path: ${{ steps.pip-cache-dir.outputs.dir }}
key: ${{ runner.os }}-pip-${{ hashFiles('requirements/edx/development.txt') }}
restore-keys: ${{ runner.os }}-pip-

- name: Install Required Python Dependencies
run: |
pip install -r requirements/pip.txt
pip install -r requirements/edx/development.txt --src ${{ runner.temp }}
pip install "django~=${{ matrix.django-version }}.0"

- name: verify unit tests count
uses: ./.github/actions/verify-tests-count
32 changes: 3 additions & 29 deletions .github/workflows/unit-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ on:

jobs:
run-tests:
if: github.repository == 'openedx/edx-platform' || github.repository == 'edx/edx-platform-private'
runs-on: [ edx-platform-runner ]
strategy:
matrix:
Expand Down Expand Up @@ -53,41 +54,14 @@ jobs:
sudo chmod -R a+rw /data/db
mongod &

- name: set settings path
run: |
echo "settings_path=$(python scripts/unit_test_shards_parser.py --shard-name=${{ matrix.shard_name }} --output settings )" >> $GITHUB_ENV

# - name: set pytest randomly option
# run: |
# echo "pytest_randomly_option=$(if [ '${{ env.module_name }}' = 'cms' ] || [ '${{ env.module_name }}' = 'common' ]; then echo '-p no:randomly'; else echo '' ; fi)" >> $GITHUB_ENV

- name: install requirements
run: |
sudo pip install -r requirements/pip.txt
sudo pip install -r requirements/edx/testing.txt
sudo pip install "django~=${{ matrix.django-version }}.0"

- name: get unit tests for shard
run: |
echo "unit_test_paths=$(python scripts/unit_test_shards_parser.py --shard-name=${{ matrix.shard_name }} )" >> $GITHUB_ENV

- name: run tests
run: |
python -Wd -m pytest -p no:randomly --ds=${{ env.settings_path }} ${{ env.unit_test_paths }}

- name: rename warnings json file
if: success()
run: |
cd test_root/log
mv pytest_warnings.json pytest_warnings_${{ matrix.shard_name }}.json

- name: save pytest warnings json file
if: success()
uses: actions/upload-artifact@v2
with:
name: pytest-warnings-json
path: |
test_root/log/pytest_warnings*.json
- name: Setup and run tests
uses: ./.github/actions/unit-tests

compile-warnings-report:
runs-on: [ edx-platform-runner ]
Expand Down
41 changes: 3 additions & 38 deletions .github/workflows/verify-gha-unit-tests-count.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ on:

jobs:
collect-and-verify:
if: github.repository == 'openedx/edx-platform' || github.repository == 'edx/edx-platform-private'
runs-on: [ edx-platform-runner ]
steps:
- name: sync directory owner
Expand All @@ -19,41 +20,5 @@ jobs:
sudo pip install -r requirements/pip.txt
sudo pip install -r requirements/edx/testing.txt

- name: collect tests from all modules
run: |
echo "root_cms_unit_tests_count=$(pytest --collect-only --ds=cms.envs.test cms/ -q | head -n -2 | wc -l)" >> $GITHUB_ENV
echo "root_lms_unit_tests_count=$(pytest --collect-only --ds=lms.envs.test lms/ openedx/ common/djangoapps/ common/lib/ -q | head -n -2 | wc -l)" >> $GITHUB_ENV

- name: get GHA unit test paths
run: |
echo "cms_unit_test_paths=$(python scripts/gha_unit_tests_collector.py --cms-only)" >> $GITHUB_ENV
echo "lms_unit_test_paths=$(python scripts/gha_unit_tests_collector.py --lms-only)" >> $GITHUB_ENV


- name: collect tests from GHA unit test shards
run: |
echo "cms_unit_tests_count=$(pytest --collect-only --ds=cms.envs.test ${{ env.cms_unit_test_paths }} -q | head -n -2 | wc -l)" >> $GITHUB_ENV
echo "lms_unit_tests_count=$(pytest --collect-only --ds=lms.envs.test ${{ env.lms_unit_test_paths }} -q | head -n -2 | wc -l)" >> $GITHUB_ENV


- name: add unit tests count
run: |
echo "root_all_unit_tests_count=$((${{ env.root_cms_unit_tests_count }}+${{ env.root_lms_unit_tests_count }}))" >> $GITHUB_ENV
echo "shards_all_unit_tests_count=$((${{ env.cms_unit_tests_count }}+${{ env.lms_unit_tests_count }}))" >> $GITHUB_ENV

- name: print unit tests count
run: |
echo CMS unit tests from root: ${{ env.root_cms_unit_tests_count }}
echo LMS unit tests from root: ${{ env.root_lms_unit_tests_count }}
echo CMS unit tests from shards: ${{ env.cms_unit_tests_count }}
echo LMS unit tests from shards: ${{ env.lms_unit_tests_count }}
echo All root unit tests count: ${{ env.root_all_unit_tests_count }}
echo All shards unit tests count: ${{ env.shards_all_unit_tests_count }}

- name: fail the check
if: ${{ env.root_all_unit_tests_count != env.shards_all_unit_tests_count }}
run: |
echo "::error title='Unit test modules in unit-test-shards.json (unit-tests.yml workflow) are outdated'::unit tests running in unit-tests
workflow don't match the count for unit tests for entire edx-platform suite, please update the unit-test-shards.json under .github/workflows
to add any missing apps and match the count. for more details please take a look at scripts/gha-shards-readme.md"
exit 1
- name: verify unit tests count
uses: ./.github/actions/verify-tests-count