Update

.gitignore

@@ -1,5 +1,6 @@
 dist
 *classes
+*.class
 target/
 build/
 build_eclipse/
@@ -47,3 +48,8 @@ tests/venv
 docs/generated/
 
 .release-settings.json
+
+kafkatest.egg-info/
+systest/
+*.swp
+clients/src/generated

CONTRIBUTING.md

@@ -1,6 +1,6 @@
 ## Contributing to Kafka
 
-*Before opening a pull request*, review the [Contributing](http://kafka.apache.org/contributing.html) and [Contributing Code Changes](https://cwiki.apache.org/confluence/display/KAFKA/Contributing+Code+Changes) pages.
+*Before opening a pull request*, review the [Contributing](https://kafka.apache.org/contributing.html) and [Contributing Code Changes](https://cwiki.apache.org/confluence/display/KAFKA/Contributing+Code+Changes) pages.
 
 It lists steps that are required before creating a PR.
 

LICENSE

@@ -201,6 +201,7 @@
    See the License for the specific language governing permissions and
    limitations under the License.
 
+------------------------------------------------------------------------------------
 This distribution has a binary dependency on jersey, which is available under the CDDL
 License as described below.
 
@@ -328,3 +329,68 @@ As between Initial Developer and the Contributors, each party is responsible for
 NOTICE PURSUANT TO SECTION 9 OF THE COMMON DEVELOPMENT AND DISTRIBUTION LICENSE (CDDL)
 
 The code released under the CDDL shall be governed by the laws of the State of California (excluding conflict-of-law provisions). Any litigation relating to this License shall be subject to the jurisdiction of the Federal Courts of the Northern District of California and the state courts of the State of California, with venue lying in Santa Clara County, California.
+
+------------------------------------------------------------------------------------
+This distribution has a binary dependency on zstd, which is available under the BSD 3-Clause License as described below.
+
+BSD License
+
+For Zstandard software
+
+Copyright (c) 2016-present, Facebook, Inc. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+ * Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+ * Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+ * Neither the name Facebook nor the names of its contributors may be used to
+   endorse or promote products derived from this software without specific
+   prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+------------------------------------------------------------------------------------
+This distribution has a binary dependency on zstd-jni, which is available under the BSD 2-Clause License
+as described below.
+
+Zstd-jni: JNI bindings to Zstd Library
+
+Copyright (c) 2015-2016, Luben Karavelov/ All rights reserved.
+
+BSD License
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice, this
+  list of conditions and the following disclaimer in the documentation and/or
+  other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

NOTICE

@@ -1,8 +1,8 @@
 Apache Kafka
-Copyright 2017 The Apache Software Foundation.
+Copyright 2019 The Apache Software Foundation.
 
 This product includes software developed at
-The Apache Software Foundation (http://www.apache.org/).
+The Apache Software Foundation (https://www.apache.org/).
 
 This distribution has a binary dependency on jersey, which is available under the CDDL
 License. The source code of jersey can be found at https://github.com/jersey/jersey/.

README.md

@@ -1,12 +1,14 @@
 Apache Kafka
 =================
-See our [web site](http://kafka.apache.org) for details on the project.
+See our [web site](https://kafka.apache.org) for details on the project.
 
-You need to have [Gradle](http://www.gradle.org/installation) and [Java](http://www.oracle.com/technetwork/java/javase/downloads/index.html) installed.
+You need to have [Gradle](https://www.gradle.org/installation) and [Java](https://www.oracle.com/technetwork/java/javase/downloads/index.html) installed.
 
-Kafka requires Gradle 3.0 or higher.
+Kafka requires Gradle 5.0 or higher.
 
-Java 7 should be used for building in order to support both Java 7 and Java 8 at runtime.
+Java 8 should be used for building in order to support both Java 8 and Java 11 at runtime.
+
+Scala 2.12 is used by default, see below for how to use a different Scala version or all of the supported Scala versions.
 
 ### First bootstrap and download the wrapper ###
     cd kafka_source_dir
@@ -17,7 +19,7 @@ Now everything else will work.
 ### Build a jar and run it ###
     ./gradlew jar
 
-Follow instructions in http://kafka.apache.org/documentation.html#quickstart
+Follow instructions in https://kafka.apache.org/documentation.html#quickstart
 
 ### Build source jar ###
     ./gradlew srcJar
@@ -43,7 +45,7 @@ Follow instructions in http://kafka.apache.org/documentation.html#quickstart
     ./gradlew cleanTest integrationTest
 
 ### Running a particular unit/integration test ###
-    ./gradlew -Dtest.single=RequestResponseSerializationTest core:test
+    ./gradlew clients:test --tests RequestResponseTest
 
 ### Running a particular test method within a unit/integration test ###
     ./gradlew core:test --tests kafka.api.ProducerFailureHandlingTest.testCannotSendToInternalTopic
@@ -52,7 +54,7 @@ Follow instructions in http://kafka.apache.org/documentation.html#quickstart
 ### Running a particular unit/integration test with log4j output ###
 Change the log4j setting in either `clients/src/test/resources/log4j.properties` or `core/src/test/resources/log4j.properties`
 
-    ./gradlew -i -Dtest.single=RequestResponseSerializationTest core:test
+    ./gradlew clients:test --tests RequestResponseTest
 
 ### Generating test coverage reports ###
 Generate coverage reports for the whole project:
@@ -64,28 +66,33 @@ Generate coverage for a single module, i.e.:
     ./gradlew clients:reportCoverage
 
 ### Building a binary release gzipped tar ball ###
-    ./gradlew clean
-    ./gradlew releaseTarGz
+    ./gradlew clean releaseTarGz
 
 The above command will fail if you haven't set up the signing key. To bypass signing the artifact, you can run:
 
-    ./gradlew releaseTarGz -x signArchives
+    ./gradlew clean releaseTarGz -x signArchives
 
 The release file can be found inside `./core/build/distributions/`.
 
 ### Cleaning the build ###
     ./gradlew clean
 
-### Running a task on a particular version of Scala (either 2.11.x or 2.12.x) ###
-*Note that if building the jars with a version other than 2.11.12, you need to set the `SCALA_VERSION` variable or change it in `bin/kafka-run-class.sh` to run the quick start.*
+### Running a task with one of the Scala versions available (2.11.x, 2.12.x or 2.13.x) ###
+*Note that if building the jars with a version other than 2.12.x, you need to set the `SCALA_VERSION` variable or change it in `bin/kafka-run-class.sh` to run the quick start.*
+
+You can pass either the major version (eg 2.12) or the full version (eg 2.12.7):
+
+    ./gradlew -PscalaVersion=2.12 jar
+    ./gradlew -PscalaVersion=2.12 test
+    ./gradlew -PscalaVersion=2.12 releaseTarGz
 
-You can pass either the major version (eg 2.11) or the full version (eg 2.11.12):
+### Running a task with all the scala versions enabled by default ###
 
-    ./gradlew -PscalaVersion=2.11 jar
-    ./gradlew -PscalaVersion=2.11 test
-    ./gradlew -PscalaVersion=2.11 releaseTarGz
+Append `All` to the task name:
 
-Scala 2.12.x requires Java 8.
+    ./gradlew testAll
+    ./gradlew jarAll
+    ./gradlew releaseTarGzAll
 
 ### Running a task for a specific project ###
 This is for `core`, `examples` and `clients`
@@ -106,15 +113,6 @@ The `eclipse` task has been configured to use `${project_dir}/build_eclipse` as
 build directory (`${project_dir}/bin`) clashes with Kafka's scripts directory and we don't use Gradle's build directory
 to avoid known issues with this configuration.
 
-### Building the jar for all scala versions and for all projects ###
-    ./gradlew jarAll
-
-### Running unit/integration tests for all scala versions and for all projects ###
-    ./gradlew testAll
-
-### Building a binary release gzipped tar ball for all scala versions ###
-    ./gradlew releaseTarGzAll
-
 ### Publishing the jar for all version of Scala and for all projects to maven ###
     ./gradlew uploadArchivesAll
 
@@ -170,7 +168,7 @@ Please note for this to work you should create/update user maven settings (typic
     ./gradlew dependencyUpdates
 
 ### Running code quality checks ###
-There are two code quality analysis tools that we regularly run, findbugs and checkstyle.
+There are two code quality analysis tools that we regularly run, spotbugs and checkstyle.
 
 #### Checkstyle ####
 Checkstyle enforces a consistent coding style in Kafka.
@@ -181,14 +179,14 @@ You can run checkstyle using:
 The checkstyle warnings will be found in `reports/checkstyle/reports/main.html` and `reports/checkstyle/reports/test.html` files in the
 subproject build directories. They are also are printed to the console. The build will fail if Checkstyle fails.
 
-#### Findbugs ####
-Findbugs uses static analysis to look for bugs in the code.
-You can run findbugs using:
+#### Spotbugs ####
+Spotbugs uses static analysis to look for bugs in the code.
+You can run spotbugs using:
 
-    ./gradlew findbugsMain findbugsTest -x test
+    ./gradlew spotbugsMain spotbugsTest -x test
 
-The findbugs warnings will be found in `reports/findbugs/main.html` and `reports/findbugs/test.html` files in the subproject build
-directories.  Use -PxmlFindBugsReport=true to generate an XML report instead of an HTML one.
+The spotbugs warnings will be found in `reports/spotbugs/main.html` and `reports/spotbugs/test.html` files in the subproject build
+directories.  Use -PxmlSpotBugsReport=true to generate an XML report instead of an HTML one.
 
 ### Common build options ###
 
@@ -200,7 +198,11 @@ The following options should be set with a `-P` switch, for example `./gradlew -
 * `showStandardStreams`: shows standard out and standard error of the test JVM(s) on the console.
 * `skipSigning`: skips signing of artifacts.
 * `testLoggingEvents`: unit test events to be logged, separated by comma. For example `./gradlew -PtestLoggingEvents=started,passed,skipped,failed test`.
-* `xmlFindBugsReport`: enable XML reports for findBugs. This also disables HTML reports as only one can be enabled at a time.
+* `xmlSpotBugsReport`: enable XML reports for spotBugs. This also disables HTML reports as only one can be enabled at a time.
+
+### Running system tests ###
+
+See [tests/README.md](tests/README.md).
 
 ### Running in Vagrant ###
 
@@ -211,4 +213,4 @@ See [vagrant/README.md](vagrant/README.md).
 Apache Kafka is interested in building the community; we would welcome any thoughts or [patches](https://issues.apache.org/jira/browse/KAFKA). You can reach us [on the Apache mailing lists](http://kafka.apache.org/contact.html).
 
 To contribute follow the instructions here:
- * http://kafka.apache.org/contributing.html
+ * https://kafka.apache.org/contributing.html