Skip to content

Comments

Create a manually started workflow for bulk run of integration tests#1937

Merged
peternied merged 5 commits intoopensearch-project:mainfrom
peternied:bulk-it
Jul 14, 2022
Merged

Create a manually started workflow for bulk run of integration tests#1937
peternied merged 5 commits intoopensearch-project:mainfrom
peternied:bulk-it

Conversation

@peternied
Copy link
Member

Description

We've been looking to increase test reliability and that has lead to many runs and re-runs of the integration test in GitHub actions. Creating a manually started workflow that runs the integration tests on both jdk11 and jdk17 10x times.

Issues Resolved

Testing

See an example of this run here.

Check List

  • New functionality includes testing
  • New functionality has been documented
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Peter Nied <petern@amazon.com>
Signed-off-by: Peter Nied <petern@amazon.com>
Signed-off-by: Peter Nied <petern@amazon.com>
@peternied peternied requested a review from a team July 13, 2022 20:15
@peternied peternied added the test fix test fixes label Jul 13, 2022

Launch IntelliJ IDEA, choose **Project from Existing Sources**, and select directory with Gradle build script (`build.gradle`).

## Running integration tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding a section on how to find the test output after CI has run!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. These are nice descriptions to add!

@cwperks
Copy link
Member

cwperks commented Jul 13, 2022

The changes look good to me. Looks like there's a minor lint issue with the integration-tests.yml file. Clever use of matrix!

@codecov-commenter
Copy link

codecov-commenter commented Jul 13, 2022

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 60.99%. Comparing base (be876c0) to head (b692052).
Report is 841 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##               main    #1937      +/-   ##
============================================
- Coverage     61.00%   60.99%   -0.02%     
  Complexity     3233     3233              
============================================
  Files           256      256              
  Lines         18088    18088              
  Branches       3224     3224              
============================================
- Hits          11035    11033       -2     
- Misses         5470     5471       +1     
- Partials       1583     1584       +1     

see 1 file with indirect coverage changes

---- 🚨 Try these New Features:

Signed-off-by: Peter Nied <petern@amazon.com>
Copy link
Member

@DarshitChanpura DarshitChanpura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great addition. Thank you @peternied !

@cliu123
Copy link
Member

cliu123 commented Jul 14, 2022

How is this new workflow tested? Can it be tested on this PR?

@peternied
Copy link
Member Author

Testing

See an example of this run here.

@cliu123 Here is an execution of this PR on my branch, 20 out of 20 passed, but you can drill in to see failures that were not blocking like our normal workflow.

Signed-off-by: Peter Nied <petern@amazon.com>
@cliu123
Copy link
Member

cliu123 commented Jul 14, 2022

p.s. Where is the button triggering this workflow? I mean I don't see the button in the Checks page. How was the example fun triggered?

@peternied peternied merged commit f153c27 into opensearch-project:main Jul 14, 2022
@peternied
Copy link
Member Author

p.s. Where is the button triggering this workflow? I mean I don't see the button in the Checks page. How was the example fun triggered?

It's only shown in the GitHub UX when main has the changes in it, since this PR has merged you should be able to follow the instructions in the dev guild to trigger it

@peternied peternied deleted the bulk-it branch July 15, 2022 17:23
jochenkressin pushed a commit to eliatra/OpenSearch-security-tests that referenced this pull request Jul 26, 2022
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 28, 2022
peternied added a commit that referenced this pull request Jul 28, 2022
…1937) (#1973)

(cherry picked from commit f153c27)

Co-authored-by: Peter Nied <petern@amazon.com>
stephen-crawford pushed a commit to stephen-crawford/security that referenced this pull request Nov 10, 2022
wuychn pushed a commit to ochprince/security that referenced this pull request Mar 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test fix test fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants