Skip to content

HDDS-11527. Avoid unnecessary duplicate build #7270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
adoroszlai merged 8 commits into from
Oct 5, 2024
Merged

HDDS-11527. Avoid unnecessary duplicate build #7270

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
d8c1ff4
HDDS-11527. Avoid unnecessary duplicate build
adoroszlai Oct 4, 2024
20be735
remove matrix build
adoroszlai Oct 4, 2024
d891f44
clarify TEST_JAVA_VERSION
adoroszlai Oct 4, 2024
06d6863
include java version in step name
adoroszlai Oct 4, 2024
34dffaf
use TEST_JAVA_VERSION in native check
adoroszlai Oct 4, 2024
c2194c0
use TEST_JAVA_VERSION in basic check
adoroszlai Oct 4, 2024
ea10202
use TEST_JAVA_VERSION in license check
adoroszlai Oct 4, 2024
119e2ed
Revert "use TEST_JAVA_VERSION in basic check"
adoroszlai Oct 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 18 additions & 25 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ on:
required: false
env:
FAIL_FAST: ${{ github.event_name == 'pull_request' }}
BUILD_JAVA_VERSION: 8 # minimum version
TEST_JAVA_VERSION: 17 # preferred version
# Minimum required Java version for running Ozone is defined in pom.xml (javac.version).
TEST_JAVA_VERSION: 17 # JDK version used by CI build and tests; should match the JDK version in apache/ozone-runner image
MAVEN_OPTS: -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.http.retryHandler.class=standard -Dmaven.wagon.http.retryHandler.count=3
OZONE_WITH_COVERAGE: ${{ github.event_name == 'push' }}
jobs:
Expand Down Expand Up @@ -104,10 +104,6 @@ jobs:
runs-on: ubuntu-20.04
timeout-minutes: 60
if: needs.build-info.outputs.needs-build == 'true'
strategy:
matrix:
java: [ 8, 17 ]
fail-fast: false
steps:
- name: Checkout project
uses: actions/checkout@v4
Expand Down Expand Up @@ -138,33 +134,31 @@ jobs:
name: ratis-jars
path: |
~/.m2/repository/org/apache/ratis
- name: Setup java
- name: Setup java ${{ env.TEST_JAVA_VERSION }}
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: ${{ matrix.java }}
java-version: ${{ env.TEST_JAVA_VERSION }}
- name: Run a full build
run: hadoop-ozone/dev-support/checks/build.sh -Pdist -Psrc -Dmaven.javadoc.skip=true ${{ inputs.ratis_args }}
env:
DEVELOCITY_ACCESS_KEY: ${{ secrets.GE_ACCESS_TOKEN }}
- name: Store binaries for tests
uses: actions/upload-artifact@v4
with:
name: ozone-bin-${{ matrix.java }}
name: ozone-bin
path: |
hadoop-ozone/dist/target/ozone-*.tar.gz
!hadoop-ozone/dist/target/ozone-*-src.tar.gz
retention-days: 1
- name: Store source tarball for compilation
uses: actions/upload-artifact@v4
if: ${{ matrix.java == env.BUILD_JAVA_VERSION }}
with:
name: ozone-src
path: hadoop-ozone/dist/target/ozone-*-src.tar.gz
retention-days: 1
- name: Store Maven repo for tests
uses: actions/upload-artifact@v4
if: ${{ matrix.java == env.BUILD_JAVA_VERSION }}
with:
name: ozone-repo
path: |
Expand Down Expand Up @@ -216,7 +210,7 @@ jobs:
name: ratis-jars
path: |
~/.m2/repository/org/apache/ratis
- name: Setup java
- name: Setup java ${{ matrix.java }}
uses: actions/setup-java@v4
with:
distribution: 'temurin'
Expand Down Expand Up @@ -265,7 +259,7 @@ jobs:
name: ratis-jars
path: |
~/.m2/repository/org/apache/ratis
- name: Setup java
- name: Setup java 8
uses: actions/setup-java@v4
with:
distribution: 'temurin'
Expand Down Expand Up @@ -313,11 +307,11 @@ jobs:
name: ratis-jars
path: |
~/.m2/repository/org/apache/ratis
- name: Setup java
- name: Setup java ${{ env.TEST_JAVA_VERSION }}
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 8
java-version: ${{ env.TEST_JAVA_VERSION }}
- name: Execute tests
run: hadoop-ozone/dev-support/checks/${{ github.job }}.sh ${{ inputs.ratis_args }}
continue-on-error: true
Expand Down Expand Up @@ -348,7 +342,7 @@ jobs:
- name: Download compiled Ozone binaries
uses: actions/download-artifact@v4
with:
name: ozone-bin-${{ env.BUILD_JAVA_VERSION }}
name: ozone-bin
- name: Untar binaries
run: |
mkdir dist
Expand Down Expand Up @@ -392,6 +386,11 @@ jobs:
name: ozone-repo
path: |
~/.m2/repository/org/apache/ozone
- name: Setup java ${{ env.TEST_JAVA_VERSION }}
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: ${{ env.TEST_JAVA_VERSION }}
- name: Execute tests
run: |
hadoop-ozone/dev-support/checks/${{ github.job }}.sh
Expand Down Expand Up @@ -425,14 +424,8 @@ jobs:
ref: ${{ needs.build-info.outputs.sha }}
- name: Download compiled Ozone binaries
uses: actions/download-artifact@v4
if: ${{ matrix.suite }} == 'MR' # until HDDS-11053 is fixed
with:
name: ozone-bin-${{ env.BUILD_JAVA_VERSION }}
- name: Download compiled Ozone binaries
uses: actions/download-artifact@v4
if: ${{ matrix.suite }} != 'MR'
with:
name: ozone-bin-${{ env.TEST_JAVA_VERSION }}
name: ozone-bin
- name: Untar binaries
run: |
mkdir -p hadoop-ozone/dist/target
Expand Down Expand Up @@ -475,7 +468,7 @@ jobs:
- name: Download compiled Ozone binaries
uses: actions/download-artifact@v4
with:
name: ozone-bin-${{ env.BUILD_JAVA_VERSION }}
name: ozone-bin
- name: Untar binaries
run: |
mkdir -p hadoop-ozone/dist/target
Expand Down Expand Up @@ -596,7 +589,7 @@ jobs:
- name: Untar binaries
run: |
mkdir -p hadoop-ozone/dist/target
tar xzvf target/artifacts/ozone-bin-${{ env.TEST_JAVA_VERSION }}/ozone*.tar.gz -C hadoop-ozone/dist/target
tar xzvf target/artifacts/ozone-bin/ozone*.tar.gz -C hadoop-ozone/dist/target
- name: Setup java ${{ env.TEST_JAVA_VERSION }}
uses: actions/setup-java@v4
with:
Expand Down