From 02618bb82882d16a5438db3fa7e43e29b83c59e4 Mon Sep 17 00:00:00 2001 From: ableegoldman Date: Fri, 6 Dec 2019 11:53:29 -0800 Subject: [PATCH 1/4] doc --- docs/streams/developer-guide/dsl-api.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/streams/developer-guide/dsl-api.html b/docs/streams/developer-guide/dsl-api.html index e54d4491c739a..31e748aa139c0 100644 --- a/docs/streams/developer-guide/dsl-api.html +++ b/docs/streams/developer-guide/dsl-api.html @@ -261,7 +261,7 @@

You must specify SerDes explicitly if the key or value types of the records in the Kafka input topics do not match the configured default SerDes. For information about configuring default SerDes, available SerDes, and implementing your own custom SerDes see Data Types and Serialization.

-

Several variants of stream exist, for example to specify a regex pattern for input topics to read from).

+

Several variants of stream exist. For example, you can specify a regex pattern for input topics to read from (note that a single nodegroup will be created for all matching topics, and therefore the number of tasks will scale with the maximum partition count rather than the total number of partitions).

Table

From 0eb794683a8b085d7ad530b9a2acb81bccce1a21 Mon Sep 17 00:00:00 2001 From: ableegoldman Date: Fri, 6 Dec 2019 12:01:21 -0800 Subject: [PATCH 2/4] javadocs --- .../java/org/apache/kafka/streams/StreamsBuilder.java | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java b/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java index 609bce5caabaa..2b36c3cba57b6 100644 --- a/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java +++ b/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java @@ -148,7 +148,9 @@ public synchronized KStream stream(final Collection topics, * deserializers as specified in the {@link StreamsConfig config} are used. *

* If multiple topics are matched by the specified pattern, the created {@link KStream} will read data from all of - * them and there is no ordering guarantee between records from different topics. + * them and there is no ordering guarantee between records from different topics. These topics will all be grouped + * into a single node group, and as such the number of tasks generated will scale with the maximum partition count + * across the matching topics, and not with the total number of partitions summed over them. *

* Note that the specified input topics must be partitioned by key. * If this is not the case it is the user's responsibility to repartition the data before any key based operation @@ -167,7 +169,9 @@ public synchronized KStream stream(final Pattern topicPattern) { * are defined by the options in {@link Consumed} are used. *

* If multiple topics are matched by the specified pattern, the created {@link KStream} will read data from all of - * them and there is no ordering guarantee between records from different topics. + * them and there is no ordering guarantee between records from different topics. These topics will all be grouped + * into a single node group, and as such the number of tasks generated will scale with the maximum partition count + * across the matching topics, and not with the total number of partitions summed over them. *

* Note that the specified input topics must be partitioned by key. * If this is not the case it is the user's responsibility to repartition the data before any key based operation From 8c7fb3e7afd9f7195599a4f467caae3c576d4932 Mon Sep 17 00:00:00 2001 From: ableegoldman Date: Fri, 6 Dec 2019 12:47:00 -0800 Subject: [PATCH 3/4] alternative phrasing --- docs/streams/developer-guide/dsl-api.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/streams/developer-guide/dsl-api.html b/docs/streams/developer-guide/dsl-api.html index 31e748aa139c0..02f2441a57db7 100644 --- a/docs/streams/developer-guide/dsl-api.html +++ b/docs/streams/developer-guide/dsl-api.html @@ -261,7 +261,7 @@

You must specify SerDes explicitly if the key or value types of the records in the Kafka input topics do not match the configured default SerDes. For information about configuring default SerDes, available SerDes, and implementing your own custom SerDes see Data Types and Serialization.

-

Several variants of stream exist. For example, you can specify a regex pattern for input topics to read from (note that a single nodegroup will be created for all matching topics, and therefore the number of tasks will scale with the maximum partition count rather than the total number of partitions).

+

Several variants of stream exist. For example, you can specify a regex pattern for input topics to read from (note that all matching topics will be part of the same input topic group, and the work will not be parallelized for different topics if subscribed to in this way).

Table

From a9a11b254cb5f3d5b37d0057d2ff46a2f5011959 Mon Sep 17 00:00:00 2001 From: ableegoldman Date: Fri, 6 Dec 2019 12:53:06 -0800 Subject: [PATCH 4/4] rephrase javadocs --- .../org/apache/kafka/streams/StreamsBuilder.java | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java b/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java index 2b36c3cba57b6..8d650d2dcefa5 100644 --- a/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java +++ b/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java @@ -148,9 +148,9 @@ public synchronized KStream stream(final Collection topics, * deserializers as specified in the {@link StreamsConfig config} are used. *

* If multiple topics are matched by the specified pattern, the created {@link KStream} will read data from all of - * them and there is no ordering guarantee between records from different topics. These topics will all be grouped - * into a single node group, and as such the number of tasks generated will scale with the maximum partition count - * across the matching topics, and not with the total number of partitions summed over them. + * them and there is no ordering guarantee between records from different topics. This also means that the work + * will not be parallelized for multiple topics, and the number of tasks will scale with the maximum partition + * count of any matching topic rather than the total number of partitions across all topics. *

* Note that the specified input topics must be partitioned by key. * If this is not the case it is the user's responsibility to repartition the data before any key based operation @@ -169,9 +169,9 @@ public synchronized KStream stream(final Pattern topicPattern) { * are defined by the options in {@link Consumed} are used. *

* If multiple topics are matched by the specified pattern, the created {@link KStream} will read data from all of - * them and there is no ordering guarantee between records from different topics. These topics will all be grouped - * into a single node group, and as such the number of tasks generated will scale with the maximum partition count - * across the matching topics, and not with the total number of partitions summed over them. + * them and there is no ordering guarantee between records from different topics. This also means that the work + * will not be parallelized for multiple topics, and the number of tasks will scale with the maximum partition + * count of any matching topic rather than the total number of partitions across all topics. *

* Note that the specified input topics must be partitioned by key. * If this is not the case it is the user's responsibility to repartition the data before any key based operation