Split gradle check into parallel check and bwc jobs

[andrross]

Run check and bwcTestSnapshots as separate matrix entries in the gradle-check GitHub Actions workflow. Remove bwcTestSnapshots as a dependency of the check task.

Check List

• Functionality includes testing.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit 67c7856)

Here are some key observations to aid the review process:

🧪 No relevant tests

🔒 No security concerns identified

✅ No TODO sections

🔀 No multiple PR themes

⚡ Recommended focus areas for review Duplicate Comments With the matrix strategy, both check and bwc jobs will independently post PR comments on success, flaky, and failure. This could result in multiple comments being posted for the same PR event, potentially causing confusion. Consider whether the create-or-update-comment action is configured to update an existing comment or always create a new one, and whether the comment body should differentiate the two jobs clearly enough to avoid noise. - name: Create Comment Success if: ${{ github.event_name == 'pull_request_target' && success() && env.result == 'SUCCESS' }} uses: peter-evans/create-or-update-comment@v5 with: issue-number: ${{ env.pr_number }} body: | :white_check_mark: Build result for **${{ matrix.label }}** (${{ env.pr_from_sha }}): [${{ env.result }}](${{ env.workflow_url }}) - name: Extract Test Failure if: ${{ github.event_name == 'pull_request_target' && env.result != 'SUCCESS' }} run: | TEST_FAILURES=`curl -s "${{ env.workflow_url }}/testReport/api/json?tree=suites\[cases\[status,className,name\]\]" | jq -r '.. | objects | select(.status=="FAILED",.status=="REGRESSION") | (.className + "." + .name)' | uniq -c | sort -n -r | head -n 10` if [[ "$TEST_FAILURES" != "" ]] then echo "test_failures<<EOF" >> $GITHUB_ENV echo "" >> $GITHUB_ENV echo "* **TEST FAILURES:**" >> $GITHUB_ENV echo '```' >> $GITHUB_ENV echo "$TEST_FAILURES" >> $GITHUB_ENV echo '```' >> $GITHUB_ENV echo "EOF" >> $GITHUB_ENV fi - name: Create Comment Flaky if: ${{ github.event_name == 'pull_request_target' && success() && env.result != 'SUCCESS' }} uses: peter-evans/create-or-update-comment@v5 with: issue-number: ${{ env.pr_number }} body: | :grey_exclamation: Build result for **${{ matrix.label }}** (${{ env.pr_from_sha }}): [${{ env.result }}](${{ env.workflow_url }}) ${{ env.test_failures }} Please review all [flaky tests](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#flaky-tests) that succeeded after retry and create an issue if one does not already exist to track the flaky failure. - name: Create Comment Failure if: ${{ github.event_name == 'pull_request_target' && failure() }} uses: peter-evans/create-or-update-comment@v5 with: issue-number: ${{ env.pr_number }} body: | :x: Build result for **${{ matrix.label }}** (${{ env.pr_from_sha }}): [${{ env.result }}](${{ env.workflow_url }}) Timeout Adequacy The timeout-minutes: 130 applies to both matrix jobs (check and bwc). The bwcTestSnapshots task may have different runtime characteristics than check. Verify that 130 minutes is sufficient for both jobs, or consider setting per-matrix timeouts if supported. timeout-minutes: 130 strategy: fail-fast: false matrix: include: - label: check gradle_check_command: 'check -Dtests.coverage=true' - label: bwc gradle_check_command: 'bwcTestSnapshots -Dbwc.buildUnreleasedFromSource=false'

[andrross]

@cwperks Is it possible to test this? Due to pull_request_target it seems to always use the version of the workflow checked into main instead of what is in the PR.

[cwperks]

@cwperks Is it possible to test this? Due to pull_request_target it seems to always use the version of the workflow checked into main instead of what is in the PR.

Unfortunately I don't think we have a way to do this unless we can authorize someone's fork to start the gradle check.

[cwperks]

@andrross ML repo has similar issues. Maybe @ylwu-amzn or @dhrubo-os would know?

[github-actions]

❌ Gradle check result for 29e867c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[dhrubo-os]

@andrross ML repo has similar issues. Maybe @ylwu-amzn or @dhrubo-os would know?

Yeah, we've hit the same issue in ml-commons. Unfortunately there's no clean way to test pull_request_target workflow changes from a fork PR — GitHub always uses the workflow from the base branch. In ml-commons we'
ve typically had to merge and validate on the next PR.

@peterzhuamazon how do you test when you raise PRs in ML-Commons in CI workflow?

[andrross]

merge and validate on the next PR

@dhrubo-os YOLO!

@cwperks I suspect that's what we'll need to do here. What do you think? All the checks will (eventually) pass on this PR, it's just that it's running the old version of the workflow.

[github-actions]

Persistent review updated to latest commit 67c7856

[github-actions]

❌ Gradle check result for 67c7856: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 67c7856: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 67c7856: SUCCESS

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.37%. Comparing base (2b89294) to head (67c7856).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #21106      +/-   ##
============================================
- Coverage     73.39%   73.37%   -0.02%     
+ Complexity    73287    73259      -28     
============================================
  Files          5921     5921              
  Lines        333798   333798              
  Branches      48124    48124              
============================================
- Hits         244992   244932      -60     
- Misses        69238    69317      +79     
+ Partials      19568    19549      -19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[cwperks]

Yea it's ok with me. IMO it's exactly what we have version control for.

The only other thing I would check is if we could somehow use act (https://github.com/nektos/act) and somehow get the authority to trigger Jenkins from local.