Skip to content

Commit

Permalink
Runs GitHub workflow unit tests in band
Browse files Browse the repository at this point in the history 
* Removes caching from GitHub test workflow to simplify.
* Removes "bad apples" check from GitHub workflow since those tests now
  pass consistently.
* Runs unit tests in band to reduce execution time from ~90 minutes to
  ~20 minutes.
* Removes the logic that ignored failing unit tests.
* Adds mocha tests to the workflow.
* Adds ci-specific scripts to simplify the commands.
  * `yarn test:jest:ci` and `yarn test:jest_integration:ci` are now used
    in the workflow.

Resolves opensearch-project#1231

Signed-off-by: Tommy Markley <[email protected]>
  • Loading branch information
Tommy Markley committed Mar 2, 2022
1 parent 9fea75d commit 9f611a6
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 114 deletions.
122 changes: 8 additions & 114 deletions .github/workflows/pr_check_workflow.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ on:
branches: [ '**', '!feature/**' ]

env:
CACHE_NAME: osd-node-modules
TEST_BROWSER_HEADLESS: 1
CI: 1
GCS_UPLOAD_PREFIX: fake
Expand All @@ -26,108 +25,40 @@ jobs:
runs-on: ubuntu-latest
name: Build and Verify
steps:
# Access a cache of set results from a previous run of the job
# This is to prevent re-running steps that were already successful since it is not native to github actions
# Can be used to verify flaky steps with reduced times
- name: Restore the cached run
uses: actions/cache@v2
with:
path: |
job_successful
linter_results
unit_tests_results
integration_tests_results
key: ${{ github.run_id }}-${{ github.job }}-${{ github.sha }}
restore-keys: |
${{ github.run_id }}-${{ github.job }}-${{ github.sha }}
- name: Get if previous job was successful
id: job_successful
run: cat job_successful 2>/dev/null || echo 'false'

- name: Get the previous linter results
id: linter_results
run: cat linter_results 2>/dev/null || echo 'default'

- name: Get the previous unit tests results
id: unit_tests_results
run: cat unit_tests_results 2>/dev/null || echo 'default'

- name: Get the previous integration tests results
id: integration_tests_results
run: cat integration_tests_results 2>/dev/null || echo 'default'

- name: Checkout code
if: steps.job_successful.outputs.job_successful != 'true'
uses: actions/checkout@v2

- name: Setup Node
if: steps.job_successful.outputs.job_successful != 'true'
uses: actions/setup-node@v2
with:
node-version-file: ".nvmrc"
registry-url: 'https://registry.npmjs.org'

- name: Setup Yarn
if: steps.job_successful.outputs.job_successful != 'true'
run: |
npm uninstall -g yarn
npm i -g [email protected]
- name: Run bootstrap
if: steps.job_successful.outputs.job_successful != 'true'
run: yarn osd bootstrap

- name: Run linter
if: steps.linter_results.outputs.linter_results != 'success'
id: linter
run: yarn lint

# Runs unit tests while limiting workers because github actions will spawn more than it can handle and crash
# Continues on error but will create a comment on the pull request if this step failed.
- name: Run unit tests
if: steps.unit_tests_results.outputs.unit_tests_results != 'success'
id: unit-tests
continue-on-error: true
run: node scripts/jest --ci --colors --maxWorkers=10
env:
SKIP_BAD_APPLES: true

- run: echo Unit tests completed unsuccessfully. However, unit tests are inconsistent on the CI so please verify locally with `yarn test:jest`.
if: steps.unit_tests_results.outputs.unit_tests_results != 'success' && steps.unit-tests.outcome != 'success'

# TODO: This gets rejected, we need approval to add this
# - name: Add comment if unit tests did not succeed
# if: steps.unit_tests_results.outputs.unit_tests_results != 'success' && steps.unit-tests.outcome != 'success'
# uses: actions/github-script@v5
# with:
# github-token: ${{ secrets.GITHUB_TOKEN }}
# script: |
# github.rest.issues.createComment({
# issue_number: context.issue.number,
# owner: context.repo.owner,
# repo: context.repo.repo,
# body: 'Unit tests completed unsuccessfully. However, unit tests are inconsistent on the CI so please verify locally with `yarn test:jest`.'
# })
run: yarn test:jest:ci

- name: Run mocha tests
id: mocha-tests
run: yarn test:mocha

- name: Run integration tests
if: steps.integration_tests_results.outputs.integration_tests_results != 'success'
id: integration-tests
run: node scripts/jest_integration --ci --colors --max-old-space-size=5120

# Set cache if linter, unit tests, and integration tests were successful then the job will be marked successful
# Sets individual results to empower re-runs of the same build without re-running successful steps.
- if: |
(steps.linter.outcome == 'success' || steps.linter.outcome == 'skipped') &&
(steps.unit-tests.outcome == 'success' || steps.unit-tests.outcome == 'skipped') &&
(steps.integration-tests.outcome == 'success' || steps.integration-tests.outcome == 'skipped')
run: echo "::set-output name=job_successful::true" > job_successful
- if: steps.linter.outcome == 'success' || steps.linter.outcome == 'skipped'
run: echo "::set-output name=linter_results::success" > linter_results
- if: steps.unit-tests.outcome == 'success' || steps.unit-tests.outcome == 'skipped'
run: echo "::set-output name=unit_tests_results::success" > unit_tests_results
- if: steps.integration-tests.outcome == 'success' || steps.integration-tests.outcome == 'skipped'
run: echo "::set-output name=integration_tests_results::success" > integration_tests_results
run: yarn test:jest_integration:ci

functional-tests:
needs: [ build-lint-test ]
runs-on: ubuntu-latest
Expand All @@ -138,76 +69,39 @@ jobs:
steps:
- run: echo Running functional tests for ciGroup${{ matrix.group }}

# Access a cache of set results from a previous run of the job
# This is to prevent re-running a CI group that was already successful since it is not native to github actions
# Can be used to verify flaky steps with reduced times
- name: Restore the cached run
uses: actions/cache@v2
with:
path: |
ftr_tests_results
key: ${{ github.run_id }}-${{ github.job }}-${{ matrix.group }}-${{ github.sha }}
restore-keys: |
${{ github.run_id }}-${{ github.job }}-${{ matrix.group }}-${{ github.sha }}
- name: Get the cached tests results
id: ftr_tests_results
run: cat ftr_tests_results 2>/dev/null || echo 'default'

- name: Checkout code
if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success'
uses: actions/checkout@v2

- name: Setup Node
if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success'
uses: actions/setup-node@v2
with:
node-version-file: ".nvmrc"
registry-url: 'https://registry.npmjs.org'

- name: Setup Yarn
if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success'
run: |
npm uninstall -g yarn
npm i -g [email protected]
- name: Get cache path
if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success'
id: cache-path
run: echo "::set-output name=CACHE_DIR::$(yarn cache dir)"

- name: Setup cache
if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success'
uses: actions/cache@v2
with:
path: ${{ steps.cache-path.outputs.CACHE_DIR }}
key: ${{ runner.os }}-yarn-${{ env.CACHE_NAME }}-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-${{ env.CACHE_NAME }}-
${{ runner.os }}-yarn-
${{ runner.os }}-
# github virtual env is the latest chrome
- name: Setup chromedriver
if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success'
run: yarn add --dev [email protected]

- name: Run bootstrap
if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success'
run: yarn osd bootstrap

- name: Build plugins
if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success'
run: node scripts/build_opensearch_dashboards_platform_plugins --no-examples --workers 10

- if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success'
- name: Run CI test group ${{ matrix.group }}
id: ftr-tests
run: node scripts/functional_tests.js --config test/functional/config.js --include ciGroup${{ matrix.group }}
env:
CI_GROUP: ciGroup${{ matrix.group }}
CI_PARALLEL_PROCESS_NUMBER: ciGroup${{ matrix.group }}
JOB: ci${{ matrix.group }}
CACHE_DIR: ciGroup${{ matrix.group }}

- if: steps.ftr-tests.outcome == 'success' || steps.ftr-tests.outcome == 'skipped'
run: echo "::set-output name=ftr_tests_results::success" > ftr_tests_results
2 changes: 2 additions & 0 deletions package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,9 @@
"opensearch": "node scripts/opensearch",
"test": "grunt test",
"test:jest": "node scripts/jest",
"test:jest:ci": "node scripts/jest --ci --colors --runInBand",
"test:jest_integration": "node scripts/jest_integration",
"test:jest_integration:ci": "node scripts/jest_integration --ci --colors --max-old-space-size=5120",
"test:mocha": "node scripts/mocha",
"test:mocha:coverage": "grunt test:mochaCoverage",
"test:ftr": "node scripts/functional_tests",
Expand Down

0 comments on commit 9f611a6

Please sign in to comment.