Skip to content

Build: Test flink modules for both scala 2.11 and 2.12 #4179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openinx merged 3 commits into from
Mar 15, 2022
Merged

Build: Test flink modules for both scala 2.11 and 2.12 #4179

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a48ae8a
Build: Test flink modules for both scala 2.11 and 2.12
openinx Feb 21, 2022
8261123
Only test the latest flink version with scala 2.11 for saving testing…
openinx Feb 22, 2022
7bb9cc4
Address comments.
openinx Mar 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 32 additions & 2 deletions .github/workflows/flink-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,37 @@ on:
- 'CONTRIBUTING.md'

jobs:
flink-tests:

# Only test the latest flink version with scala 2.11 for saving testing time.
flink-scala-2-11-tests:
runs-on: ubuntu-latest
strategy:
matrix:
jvm: [ 8, 11 ]
flink: [ '1.14' ]
env:
SPARK_LOCAL_IP: localhost
steps:
- uses: actions/checkout@v2
- uses: actions/setup-java@v1
with:
java-version: ${{ matrix.jvm }}
- uses: actions/cache@v2
with:
path: ~/.gradle/caches
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle') }}
restore-keys: ${{ runner.os }}-gradle
- run: echo -e "$(ip addr show eth0 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)\t$(hostname -f) $(hostname -s)" | sudo tee -a /etc/hosts
- run: ./gradlew -DsparkVersions= -DhiveVersions= -DflinkVersions=${{ matrix.flink }} :iceberg-flink:iceberg-flink-${{ matrix.flink }}:check :iceberg-flink:iceberg-flink-runtime-${{ matrix.flink }}:check -DscalaVersion=2.11 -DknownScalaVersions=2.11 -Pquick=true -x javadoc

@openinx openinx Feb 22, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR should merge before this PR : #4193 . Because if we don't merge the #4193 then this artifact name iceberg-flink-runtime-1.14 is not correct. (It is iceberg-flink-runtime-1.14_2.12)

- uses: actions/upload-artifact@v2
if: failure()
with:
name: test logs
path: |
**/build/testlogs

# Test all flink versions with scala 2.12 for general validation.
flink-scala-2-12-tests:
runs-on: ubuntu-latest
strategy:
matrix:
Expand All @@ -79,7 +109,7 @@ jobs:
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle') }}
restore-keys: ${{ runner.os }}-gradle
- run: echo -e "$(ip addr show eth0 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)\t$(hostname -f) $(hostname -s)" | sudo tee -a /etc/hosts
- run: ./gradlew -DsparkVersions= -DhiveVersions= -DflinkVersions=${{ matrix.flink }} :iceberg-flink:iceberg-flink-${{ matrix.flink }}:check :iceberg-flink:iceberg-flink-runtime-${{ matrix.flink }}:check -Pquick=true -x javadoc
- run: ./gradlew -DsparkVersions= -DhiveVersions= -DflinkVersions=${{ matrix.flink }} :iceberg-flink:iceberg-flink-${{ matrix.flink }}:check :iceberg-flink:iceberg-flink-runtime-${{ matrix.flink }}:check -DscalaVersion=2.12 -DknownScalaVersions=2.12 -Pquick=true -x javadoc

@rdblue rdblue Mar 13, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

knownScalaVersions shouldn't need to change, right? That is the list of versions that are supported by the build.

@openinx openinx Mar 14, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable to me, I've just reverted this change.

- uses: actions/upload-artifact@v2
if: failure()
with:
Expand Down