Conversation
…pache#7723) This patch fixes a bug in `SocketServer` in the expiration of connections which have not re-authenticated quickly enough. Previously these connections were left hanging, but now they are properly closed and cleaned up. This was one cause of the flaky test failures in `EndToEndAuthorizationTest.testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl`. Reviewers: Jason Gustafson<jason@confluent.io>, Rajini Sivaram <rajinisivaram@googlemail.com>
Convert StreamStreamJoinIntegrationTest to TTD for more stable testing. Reviewers: Matthias J. Sax <mjsax@apache.org>
…forever (apache#7763) If a non-consumer group is specified in `describeConsumerGroup`, the future will hang indefinitely because the future callback is never completed. This patch fixes the problem by completing the future exceptionally with an `IllegalArgumentException`. Reviewers: Stanislav Kozlovski <stanislav_kozlovski@outlook.com>, Jason Gustafson <jason@confluent.io>
…bie Connect worker rejoins the group (apache#7771) Check connectivity with broker coordinator in intervals and stop tasks if coordinator is unreachable by setting `assignmentSnapshot` to null and resetting rebalance delay when there are no lost tasks. And, because we're now sometimes setting `assignmentSnapshot` to null and reading it from other methods and thread, made this member volatile and used local references to ensure consistent reads. Adapted existing unit tests to verify additional debug calls, added more specific log messages to `DistributedHerder`, and added a new integration test that verifies the behavior when the brokers are stopped and restarted only after the workers lose their heartbeats with the broker coordinator. Author: Konstantine Karantasis <konstantine@confluent.io> Reviewers: Greg Harris <gregh@confluent.io>, Randall Hauch <rhauch@gmail.com>
Reviewed-By: Jason Gustafson <jason@confluent.io>
Reviewers: Bruno Cadonna <bruno@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
…sts (apache#7789) Author: Randall Hauch <rhauch@gmail.com> Reviewers: Nigel Liang <nigel@nigelliang.com>, Roesler <john@confluent.io>
Author: Randall Hauch <rhauch@gmail.com> Reviewer: Ismael Juma <ismael@confluent.io>
…uring reassignment (apache#7795) KIP-320 improved fetch semantics by adding leader epoch validation. This relies on reliable propagation of leader epoch information from the controller. Unfortunately, we have encountered a bug during partition reassignment in which the leader epoch in the controller context does not get properly updated. This causes UpdateMetadata requests to be sent with stale epoch information which results in the metadata caches on the brokers falling out of sync. This bug has existed for a long time, but it is only a problem due to the new epoch validation done by the client. Because the client includes the stale leader epoch in its requests, the leader rejects them, yet the stale metadata cache on the brokers prevents the consumer from getting the latest epoch. Hence the consumer cannot make progress while a reassignment is ongoing. Although it is straightforward to fix this problem in the controller for the new releases (which this patch does), it is not so easy to fix older brokers which means new clients could still encounter brokers with this bug. To address this problem, this patch also modifies the client to treat the leader epoch returned from the Metadata response as "unreliable" if it comes from an older version of the protocol. The client in this case will discard the returned epoch and it won't be included in any requests. Also, note that the correct epoch is still forwarded to replicas correctly in the LeaderAndIsr request, so this bug does not affect replication. Reviewers: Jun Rao <junrao@gmail.com>, Stanislav Kozlovski <stanislav_kozlovski@outlook.com>, Ismael Juma <ismael@juma.me.uk>
…ibuted system tests (apache#7806)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2.4.0-rc4
…on (apache#7793) Updates the HTML docs and the javadoc. Reviewers: John Roesler <vvcephei@apache.org>
Fixed a small typo on the Processor API page of the Kafka Streams developer guide docs. ("buildeer" changed to "builder")
Reviewers: Bill Bejeck <bbejeck@gmail.com>
The AK Streams architecture docs should explain how the maximum parallelism is determined Reviewers: Bill Bejeck <bbejeck@gmail.com>
…7696) Reviewers: Jason Gustafson <jason@confluent.io>
…che#7117) Fixes case where multiple children merged from a key-changing node causes an NPE. Reviewers: Matthias J. Sax <mjsax@apache.org>, Boyang Chen <boyang@confluent.io>
Reviewers: Vikas Singh <vikas@confluent.io>, Jason Gustafson <jason@confluent.io>, Shailesh Panwar <spanwar@confluent.io> (cherry picked from commit 7e36865)
Author: Manikumar Reddy <manikumar.reddy@gmail.com> Reviewers: Ismael Juma <ismael@juma.me.uk>, Magnus Edenhill <magnus@edenhill.se>, Colin Patrick McCabe <cmccabe@apache.org> Closes apache#7818 from omkreddy/zk-note
) For scenarios where the incoming traffic of all input partitions are small, there's a pitfall that the enforced processing timer is not reset after we have enforce processed ALL records. The fix itself is pretty simple: we just reset the timer when there's no buffered records. Reviewers: Javier Holguera <javier.holguera@gmail.com>, Boyang Chen <boyang@confluent.io>, Bill Bejeck <bill@confluent.io> fix unit test
…e#7845) Reviewers: Matthias J. Sax <mjsax@apache.org>
…in KStream (apache#7520) Reviewers: John Roesler <john@confluent.io>, Bill Bejeck <bbejeck@gmail.com>
…d multiple transformer instances being created. (apache#7685) This is a followup PR for apache#7520 to address issue of multiple calls to get() as it was pointed out by @bbejeck in apache#7520 (comment) Reviewers: Bill Bejeck <bbejeck@gmail.com>
Minor followup to apache#7825, should be cherry-picked to 2.4 Reviewers: Bill Bejeck <bbejeck@gmail.com>
…rrors (apache#7840) Allow transaction metadata to be reloaded, even if it already exists as of a previous epoch. This helps with cases where a previous become-follower transition failed to unload corresponding metadata. Reviewers: Jun Rao <junrao@gmail.com>, Jason Gustafson <jason@confluent.io>
…p v3 and below (apache#7753) The v3 JoinGroup logic does not properly complete the initial heartbeat for new members, which then expires after the static 5 minute timeout if the member does not rejoin. The core issue is in the `shouldKeepAlive` method, which returns false when it should return true because of an inconsistent timeout check. Reviewers: Jason Gustafson <jason@confluent.io>
…ilure (apache#7682) This patch fixes an NPE in `DefaultMetadataUpdater` due to an inconsistency in event expectations. Whenever there is an authentication failure, we were treating it as a failed update even if was from a separate connection from an inflight metadata request. This patch fixes the problem by making the `MetadataUpdater` api clearer in terms of the events that are handled. Reviewers: Stanislav Kozlovski <stanislav_kozlovski@outlook.com>, Rajini Sivaram <rajinisivaram@googlemail.com>
…bstone keys (apache#7820) Fixes an NPE when UserData in a member's subscription is null and adds similar checks for transaction log parser. Also modifies the output logic so that we show the keys of tombstones for both group and transaction metadata. Reviewers: David Arthur <mumrah@gmail.com>
TICKET = KAFKA-2561 LI_DESCRIPTION = Make Kafka capable of using BoringSSL for encryption. EXIT_CRITERIA = TICKET [KAFKA-2561]
TICKET = KAFKA-8249 LI_DESCRIPTION = kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignment of the deleted topic never comes to completion after topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller not to resume topic deletion for online broker and not to register broker notification handler during onBrokerStartup. The fix cleans up pending partition reassignment during topic deletion. EXIT_CRITERIA = TICKET [KAFKA-8249]
… add AdminZkClient#addPartitions (#60) TICKET = LI_DESCRIPTION = 1) Remove "KafkaZkClient#setOrCreatePartitionReassignment(reassignment: collection.Map[TopicPartition, Seq[Int]]): Unit" function, which has been temporarily added to preserve the dependency of Cruise Control on this API. Since Cruise Control no longer depends on this API (see linkedin/cruise-control@df52a90#diff-ab474ca3213f2c014b31470c9e344f36L91), the corresponding EXIT CRITERIA is satisfied and we can remove this API. 2) Add "AdminZkClient#addPartitions(topic: String, existingAssignment: Map[Int, Seq[Int]], allBrokers: Seq[BrokerMetadata], numPartitions: Int = 1, replicaAssignment: Option[Map[Int, Seq[Int]]] = None, validateOnly: Boolean = false): Map[Int, Seq[Int]]". This method ensures that the API consistency is maintained with the upstream Kafka -- i.e. Originally this API exists in Apache Kafka 2.3, but has been removed from LinkedIn Kafka 2.3. EXIT_CRITERIA = TICKET [KAFKA-8527] * [LI-HOTFIX] Address the feedback. TICKET = LI_DESCRIPTION = EXIT_CRITERIA = MANUAL [""]
…cessary resource use during index operations. (#64) TICKET = LI_DESCRIPTION = PR apache#6385 has added several debug and trace statements in AbstractIndex.scala, OffsetIndex.scala, and TimeIndex.scala. However, not guarding these logging statements yields waste of resources during index operations. This patch adds log guarding to avoid this resource waste. EXIT_CRITERIA = MANUAL [""]
… issue (#69) TICKET = LI_DESCRIPTION = DynamicBrokerConfig holds a reference to DynamicConfig.Broker.ClusterLevelConfigs. During DynamicConfig object initialization, DynamicBrokerConfig object is created if it was not created before. kafka-configs.sh script (ConfigCommand.scala) tries to create DynamicConfig object directly, which in turns creates DynamicBrokerConfig object. This fix makes sure that when initializing DynamicBrokerConfig object, DynamicConfig.Broker.ClusterLevelConfigs should have already been defined. EXIT_CRITERIA = MANUAL [""]
TICKET = LI_DESCRIPTION = Add the OneAboveMinIsrCounter metrics to help determining the good timing of broker rolling bounce. EXIT_CRITERIA = MANUAL [""]
TICKET = LI_DESCRIPTION = EXIT_CRITERIA = MANUAL [""]
…ghWatermarks, patch 1/2 (apache#6741) TICKET = LI_DESCRIPTION = This PR works to improve high watermark checkpointing performance. `ReplicaManager.checkpointHighWatermarks()` was found to be a major contributor to GC pressure, especially on Kafka clusters with high partition counts and low throughput. Added a JMH benchmark for `checkpointHighWatermarks` which establishes a performance baseline. The parameterized benchmark was run with 100, 1000 and 2000 topics. Modified `ReplicaManager.checkpointHighWatermarks()` to avoid extra copies and cached the Log parent directory Sting to avoid frequent allocations when calculating `File.getParent()`. A few clean-ups: * Changed all usages of Log.dir.getParent to Log.parentDir and Log.dir.getParentFile to Log.parentDirFile. * Only expose public accessor for `Log.dir` (consistent with `Log.parentDir`) * Removed unused parameters in `Partition.makeLeader`, `Partition.makeFollower` and `Partition.createLogIfNotExists`. Benchmark results: | Topic Count | Ops/ms | MB/sec allocated | |-------------|---------|------------------| | 100 | + 51% | - 91% | | 1000 | + 143% | - 49% | | 2000 | + 149% | - 50% | Reviewers: Lucas Bradstreet <lucas@confluent.io>. Ismael Juma <ismael@juma.me.uk> Co-authored-by: Gardner Vickers <gardner@vickers.me> Co-authored-by: Ismael Juma <ismael@juma.me.uk> EXIT_CRITERIA = MANUAL [""]
…ng unnecessary loading of indexes (apache#8346) TICKET = LI_DESCRIPTION = KAFKA-7283 enabled lazy mmap on index files by initializing indices on-demand rather than performing costly disk/memory operations when creating all indices on broker startup. This helped reducing the startup time of brokers. However, segment indices are still created on closing segments, regardless of whether they need to be closed or not. This is a cleaned up version of apache#7900, which was submitted by @efeg. It eliminates unnecessary disk accesses and memory map operations while deleting, renaming or closing offset and time indexes. In a cluster with 31 brokers, where each broker has 13K to 20K segments, @efeg and team observed up to 2 orders of magnitude faster LogManager shutdown times - i.e. dropping the LogManager shutdown time of each broker from 10s of seconds to 100s of milliseconds. To avoid confusion between `renameTo` and `setFile`, I replaced the latter with the more restricted updateParentDir` (it turns out that's all we need). Reviewers: Jun Rao <junrao@gmail.com>, Andrew Choi <a24choi@edu.uwaterloo.ca> Co-authored-by: Adem Efe Gencer <agencer@linkedin.com> Co-authored-by: Ismael Juma <ismael@juma.me.uk> EXIT_CRITERIA = HASH [222726d]
TICKET = KAFKA-7186 LI_DESCERIPTION = Currently the controller is creating a separate copy of UpdateMetadataRequest for each broker in the cluster, which may cause OOMs upon controller startup. This patch is the first step toward making the UpdateMetadataRequests cacheable. We change the broker-specific epoch in control requests to a common maximum broker epoch seen in all brokers, such that all UpdateMetadataRequests have the same payload. EXIT_CRITERIA = KAFKA-7186
…tocol version >= 2.3-IV2 (#75) TICKET = KAFKA-7186 LI_DESCERIPTION = Cache the UpdateMetadataRequest when using the latest inter broker protocol version EXIT_CRITERIA = KAFKA-7186
) TICKET = LI_DESCRIPTION = This Patch separates Kafka Controller Node from non-controller (data) Node. Summary of changes: 1)Add “preferred.controller” broker config (its type is boolean) to indicate whether a broker is used as a preferred controller node. 2)Add “allow.preferred.controller.fallback” broker config that determines whether a non-preferred controller node (data node) is allowed to become controller in the absence of preferred controller nodes. If “allow.preferred.controller.fallback” is set to “false”, a non-preferred controller node never tries to become controller, and only preferred controller nodes can be elected as controller. If “allow.preferred.controller.fallback” is set to “true”, a non-preferred controller node tries to elect itself as controller only in the absence of preferred controller node and a non-preferred active controller resigns as controller when it detects any live preferred controller. 3)Add ActivePreferredControllerCount metric to track whether the current broker is preferred controller node and is elected as Kafka controller 4)Add StandbyPreferredControllerCount metric to track whether the current broker is preferred controller node and is not elected as controller. EXIT_CRITERIA = MANUAL [""]
…elds (#77) TICKET = LI_DESCERIPTION = committing to our fork until upstream picks this up EXIT_CRITERIA = when KAFKA-9855 is accepted upstream (specifically - apache#8472) Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
…PartitionException upon topic creation and deletion. (#80) TICKET = LI_DESCRIPTION = When does UnknownTopicOrPartitionException typically occur? * Upon a topic creation, a follower broker of a new partition starts replica fetcher before the prospective leader broker of the new partition receives the leadership information from the controller. Apache Kafka has a an open issue about this (see KAFKA-6221) * Upon a topic deletion, a follower broker of a to-be-deleted partition starts replica fetcher after the leader broker of the to-be-deleted partition processes the deletion information from the controller. * As expected, clusters with frequent topic creation and deletion report UnknownTopicOrPartitionException with relatively higher frequency. What is the impact? * Exception tracking systems identify the error logs with UnknownTopicOrPartitionException as an exception. This results in a lot of noise for a transient issue that is expected to recover by itself and a natural process in Kafka due to its asynchronous state propagation. Why not move it to a lower than warn-level log? * Despite typically being a transient issue, UnknownTopicOrPartitionException may also indicate real issues if it doesn't fix itself after a short period of time. To ensure detection of such scenarios, this PR sets the log level to warn. EXIT_CRITERIA = TICKET [KAFKA-6221]
…rings in MetricName (#83) TICKET = LI_DESCRIPTION = SelectorMetrics has a per-connection metrics, which means the number of MetricName objects and the strings associated with it (such as group name and description) grows with the number of connections in the client. This overhead of duplicate string objects is amplified when there are multiple instances of kafka clients within the same JVM. This patch address some of the memory overhead by making metricGrpName a constant and introducing a new constant perConnectionMetricGrpName. Additionally, the strings for metric name and description in createMeter have been interned since there are about 8 types of meter metric for every single client. EXIT_CRITERIA = TICKET [KAFKA-10012]
…ntIntegrationTest. TICKET = LI_DESCRIPTION = EXIT_CRITERIA = MANUAL [""]
TICKET = LI_DESCRIPTION = LIKAFKA-9737 Broker-side changes for pass-through solution EXIT_CRITERIA = MANUAL ["If we no longer need pass-through support in Kafka. Only known user is KMM for venice; BrooklinMM has not enabled this everywhere."]
TICKET = LI_DESCRIPTION = Add recompressionRate sensor RB=1310804 G=Kafka-Code-Reviews R=mgharat A=jkoshy EXIT_CRITERIA = MANUAL ["None"]
TICKET = LI_DESCRIPTION = 1. Fix broken travis test by removing the "gradle wrapper" step 2. Always run :clients:test and :core:test for each PR 3. Cleaned up the step of uploading archives EXIT_CRITERIA = MANUAL [""]
…etricsAreGreedilyRegistered TICKET = KAFKA-9786 LI_DESCRIPTION = The test is flaky since gradle can use the same jvm to run multiple tests concurrently EXIT_CRITERIA = KAFKA-9786
…ausedByBecomeLeader TICKET = KAFKA-9750 LI_DESCRIPTION = EXIT_CRITERIA = Once we upgrade to 2.5.0 which has the fix in KAFKA-9750
TICKET = LI_DESCERIPTION = This patch is to resolve the following failure that occured during ELR io.netty:netty-handler:4.1.48.Final: Vulnerability found and is blocked by oss-canary: vulnerability: netty-handler is vulnerable to man-in-the-middle attacks. The library uses an SSLEngine that does not verify certificate hostnames when establishing connections with clients by default. This allows an attacker to potentially intercept and modify network traffic in a successful man-in-the-middle attack. Upon investigation, the netty-handler version 4.1.48.Final is pulled by zookeeper version 3.5.8 +--- org.apache.zookeeper:zookeeper:3.5.8 | +--- io.netty:netty-handler:4.1.48.Final EXIT_CRITERIA = when the transitive dependency version of netty-handler is above 4.1.48.Final
…request and response pair (#88) TICKET = LI_DESCRIPTION = The method Observer.describeRequestAndResponse is used to generate request/response information to log when the observer throws an exception. The generated request/response information will be logged together with the exception thrown by the observer. Logging the connection ID along with the exception will help us understand how the value of connection ID plays a part in the exception and help us root cause the problem. EXIT_CRITERIA = Never since this is a linkedin internal change
…ntegrationTest.testSynchronousAuthorizerAclUpdatesBlockRequestThreads (apache#7918) The test blocks requests threads while sending the ACL update requests and occasionally hits request timeout. Updated the test to tolerate timeouts and retry the request for that case. Added an additional check to verify that the requests threads are unblocked when the semaphore is released, ensuring that the timeout is not due to blocked threads. Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
…debug-level. (#89) TICKET = LI_DESCRIPTION = Info-level logs cause a line to be logged for each partition seek. This leads to an excessive number of logs in Kafka consumer users with frequent seek operations. Pre-2.3 versions of Kafka use debug-level logs for seek operations. This patch brings this behavior back. EXIT_CRITERIA = TICKET [KAFKA-8883]
This adds additional information to the quota logs, the bounds which are configured are also logged. When enforcement occurs there is an existing log, this is changed to info (from debug). Finally, many of the quota logs contain usless information (NaN) or (0.0) value entries. Logging for these entries is skipped entirely. TICKET = LI_DESCRIPTION = LIKAFKA-32112, EXIT_CRITERIA = MANUAL this is not going to merged with upstream since it change levels and I assume the original changes to dump all the quota metrics is not in upstream this too will probably be rejected from upstream.
#94) [LI-HOTFIX] Disable auto topic creation triggered by fetching metadata for all topics TICKET = KAFKA-10606 LI_DESCRIPTION = LIKAFKA-32857 EXIT_CRITERIA = Once we cherry-pick the fix for KAFKA-10606
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[LI-HOTFIX] Reject stale metadata from different cluster.
This patch adds validation during metadata update to reject md from different cluster to avoid clients consuming/producing to unexpected cluster when all brokers in current cluster are removed and added to a different cluster.
This patch also fix tests where kafka.apache.org resolves to 3 IP addresses, from upstream apache#9294, current DNS tests w.r.t kafka.apache.org are failing.
TICKET =
LI_DESCRIPTION = LIKAFKA-32543,
EXIT_CRITERIA = MANUAL this is not going to merged with upstream