Skip to content

MINOR: Add verification step for Streams archetype to Jenkins build #6431

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bbejeck merged 4 commits into from
Mar 21, 2019
Merged

MINOR: Add verification step for Streams archetype to Jenkins build #6431

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fe76355
MINOR: Add verification step for Streams archetype to Jenkins build
mjsax Mar 11, 2019
e3fa62e
Remove archetype fix
mjsax Mar 14, 2019
7f21b28
Fix typos
mjsax Mar 14, 2019
7fc213e
Github comments
mjsax Mar 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions docs/streams/tutorial.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,12 @@ <h4><a id="tutorial_maven_setup" href="#tutorial_maven_setup">Setting up a Maven
</pre>

<p>
The <code>pom.xml</code> file included in the project already has the Streams dependency defined,
and there are already several example programs written with Streams library under <code>src/main/java</code>.
The <code>pom.xml</code> file included in the project already has the Streams dependency defined.
Note, that the generated <code>pom.xml</code> targets Java 8, and does not work with higher Java versions.
</p>

<p>
There are already several example programs written with Streams library under <code>src/main/java</code>.
Since we are going to start writing such programs from scratch, we can now delete these examples:
</p>

Expand Down
42 changes: 42 additions & 0 deletions jenkins.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,45 @@
./gradlew unitTest integrationTest \
--profile --no-daemon --continue -PtestLoggingEvents=started,passed,skipped,failed "$@" \
|| { echo 'Test steps failed'; exit 1; }

# Verify that Kafka Streams archetype compiles
if [ $JAVA_HOME = "/home/jenkins/tools/java/latest11" ] ; then

@mjsax mjsax Mar 14, 2019

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The created archetype, only works for Java8 -- we would need to create a different archetype for Java11.

What we could do, is to include the Java11 setting as comments in the archetype (thoughts)?

@vvcephei vvcephei Mar 14, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could do that, or we can just publish another archetype for use with Java 11+. Then, when we drop support for Java 8, it's as simple as just deleting the Java 8 archetype.

@mjsax mjsax Mar 14, 2019

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how we could have two archetypes at the same time? I guess we would need to encode the Java version in the archetype name?

-DarchetypeGroupId=org.apache.kafka
-DarchetypeArtifactId=streams-quickstart-java   <--- change this to `streams-quickstart-java-8` and `streams-quickstart-java-11` ?
-DarchetypeVersion=$version

@vvcephei vvcephei Mar 14, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that would be the way to do it.

@mjsax mjsax Mar 14, 2019

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guozhangwang Can/should we do this in this PR? Or would this need a KIP? Or just create a JIRA for a follow up PR?

@guozhangwang guozhangwang Mar 15, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can create a JIRA ticket for a follow-up PR (this would need a KIP, yes).

@mjsax mjsax Mar 15, 2019

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created https://issues.apache.org/jira/browse/KAFKA-8116

echo "Skipping Kafka Streams archetype test for Java 11"
exit 0
fi

./gradlew streams:install clients:install connect:json:install connect:api:install \
|| { echo 'Could not install kafka-streams.jar (and dependencies) locally`'; exit 1; }

version=`grep "^version=" gradle.properties | cut -d= -f 2` \
|| { echo 'Could not get version from `gradle.properties`'; exit 1; }

cd streams/quickstart \
|| { echo 'Could not change into directory `streams/quickstart`'; exit 1; }

# variable $MAVEN_LATEST__HOME is provided by Jenkins (see build configuration)
mvn=$MAVEN_LATEST__HOME/bin/mvn

$mvn clean install -Dgpg.skip \
|| { echo 'Could not `mvn install` streams quickstart archetype'; exit 1; }

mkdir test-streams-archetype && cd test-streams-archetype \
|| { echo 'Could not create test directory for stream quickstart archetype'; exit 1; }

echo "Y" | $mvn archetype:generate \
-DarchetypeCatalog=local \
-DarchetypeGroupId=org.apache.kafka \
-DarchetypeArtifactId=streams-quickstart-java \
-DarchetypeVersion=$version \
-DgroupId=streams.examples \
-DartifactId=streams.examples \
-Dversion=0.1 \
-Dpackage=myapps \
|| { echo 'Could not create new project using streams quickstart archetype'; exit 1; }

cd streams.examples \
|| { echo 'Could not change into directory `streams.examples`'; exit 1; }

$mvn compile \
|| { echo 'Could not compile streams quickstart archetype project'; exit 1; }