Create a manually started workflow for bulk run of integration tests

.github/workflows/integration-tests.yml

@@ -0,0 +1,40 @@
+name: Bulk Integration Test
+
+on: [
+  workflow_dispatch,
+  # Uncomment below to automatically run on pushes to branches
+  # push,
+  ]
+
+env:
+  GRADLE_OPTS: -Dhttp.keepAlive=false
+
+jobs:
+  bulk-integration-test-run:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        jdk: [11, 17]
+        test-run: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
+
+    steps:
+    - uses: actions/setup-java@v2
+      with:
+        distribution: temurin # Temurin is a distribution of adoptium
+        java-version: ${{ matrix.jdk }}
+
+    - uses: actions/checkout@v2
+
+    - run: OPENDISTRO_SECURITY_TEST_OPENSSL_OPT=true ./gradlew test
+
+    - uses: actions/upload-artifact@v3
+      if: always()
+      with:
+        name: ${{ matrix.jdk }}-${{ matrix.test-run }}-reports
+        path: |
+          ./build/reports/
+
+    - name: check archive for debugging
+      if: always()
+      run: echo "Check the artifact ${{ matrix.jdk }}-${{ matrix.test-run }}-reports.zip for detailed test results"

DEVELOPER_GUIDE.md

@@ -6,6 +6,8 @@ So you want to contribute code to this project? Excellent! We're glad you're her
     - [Native platforms](#native-platforms)
   - [Building](#building)
   - [Using IntelliJ IDEA](#using-intellij-idea)
+  - [Running integration tests](#running-integration-tests)
+    - [Bulk test runs](#bulk-test-runs)
   - [Submitting Changes](#submitting-changes)
   - [Backports](#backports)
 
@@ -135,6 +137,15 @@ curl -XGET https://localhost:9200/_plugins/_security/authinfo -u 'admin:admin' -
 
 Launch IntelliJ IDEA, choose **Project from Existing Sources**, and select directory with Gradle build script (`build.gradle`).
 
+## Running integration tests
+
+Locally these can be run with `./gradlew test` with detailed results being avaliable at `${project-root}/build/reports/tests/test/index.html`, or run through an IDEs JUnit test runner.
+
+Integration tests are automatically run on all pull requests for all supported versions of the JDK.  These must pass for change(s) to be merged.  Detailed logs of these test results are avaliable by going to the GitHub action workflow's summary view and downloading the associated jdk version run of the tests, after extracting this file onto your local machine integration tests results are at `./tests/tests/index.html`.
+
+### Bulk test runs
+To collect reliability data on test runs there is a manual GitHub action workflow called `Bulk Integration Test`.  The workflow is started for a branch on this project or in a fork by going to [GitHub action workflows](https://github.com/opensearch-project/security/actions/workflows/integration-tests.yml) and selecting `Run Workflow`.
+
 ## Submitting Changes
 
 See [CONTRIBUTING](CONTRIBUTING.md).