KAFKA-19535: add integration tests for DescribeProducersOptions#brokerId#6
Closed
apalan60 wants to merge 19 commits into
Closed
KAFKA-19535: add integration tests for DescribeProducersOptions#brokerId#6apalan60 wants to merge 19 commits into
apalan60 wants to merge 19 commits into
Conversation
Update the KRaft dynamic voter set documentation. In Kafka 4.1, we introduced a powerful new feature that enables seamless migration from a static voter set to a dynamic voter set. Reviewers: Ken Huang <s7133700@gmail.com>, TengYao Chi <kitingiao@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
### Case 1: no --kafka-url and --kafka-archive
Should fail. One of argument (--kafka-url/--kafka-archive) is required.
```
> python docker_build_test.py apache/kafka --image-tag KAFKA-18841
--image-type jvm --build
usage: docker_build_test.py [-h] [--image-tag TAG] [--image-type
{jvm,native}] [--build] [--test] (--kafka-url KAFKA_URL |
--kafka-archive KAFKA_ARCHIVE) image
docker_build_test.py: error: one of the arguments --kafka-url/-u
--kafka-archive/-a is required
```
### Case 2: --kafka-url with native
```
> python docker_build_test.py apache/kafka --image-tag KAFKA-18841
--image-type native --kafka-url
https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/kafka_2.13-4.0.0.tgz
--build
```
### Case 3: --karka-url with jvm
```
> python docker_build_test.py apache/kafka --image-tag KAFKA-18841
--image-type jvm --kafka-url
https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/kafka_2.13-4.0.0.tgz
--build
```
### Case 4: --kafka-archive with native
```
> ./gradlew clean releaseTarGz
> cd docker
> python docker_build_test.py apache/kafka --image-tag KAFKA-18841
--image-type native --kafka-archive
</absolute/path/to/core/build/distributions/kafka_2.13-4.1.0-SNAPSHOT.tgz>
--build
```
### Case 5: --kafka-archive with jvm
```
> ./gradlew clean releaseTarGz
> cd docker
> python docker_build_test.py apache/kafka --image-tag KAFKA-18841
--image-type jvm --kafka-archive
</absolute/path/to/core/build/distributions/kafka_2.13-4.1.0-SNAPSHOT.tgz>
--build
```
Reviewers: Vedarth Sharma <vesharma@confluent.io>, Chia-Ping Tsai
<chia7712@gmail.com>, TengYao Chi <frankvicky@apache.org>
---------
Signed-off-by: PoAn Yang <payang@apache.org>
…tionMetadata with non-empty topic partitions (apache#20370) This is followup PR for apache#19699. * Update TransactionLog#readTxnRecordValue to initialize TransactionMetadata with non-empty topic partitions * Update `TxnTransitMetadata` comment, because it's not immutable. Reviewers: TengYao Chi <kitingiao@gmail.com>, Justine Olshan <jolshan@confluent.io>, Kuan-Po Tseng <brandboat@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
A bunch of cleanups in the release scripts Reviewers: Luke Chen <showuon@gmail.com>
…0288) Offline migration essentially preserves offsets and nothing else. So effectively write tombstones for classic group type when a streams heartbeat is sent to with the group ID of an empty classic group, and write tombstones for the streams group type when a classic consumer attempts to join with a group ID of an empty streams group. Reviewers: Bill Bejeck <bbejeck@apache.org>, Sean Quah <squah@confluent.io>, Dongnuo Lyu <dlyu@confluent.io>
…ader is unavailable (apache#20271) This PR applies the same partition leader check for `StreamsGroupCommand` as `ShareGroupCommand` and `ConsumerGroupCommand` to avoid the command execution timeout. Reviewers: Lucas Brutschy <lucasbru@apache.org>
Add Unit Tests for an empty follower fetch for various Leader states. | TieredStorage Enabled | Leader Log Start Offset | Leader Local Log Start Offset | Leader Log End Offset | Remarks | |-----------------------|-------------------------|--------------------------------|-----------------------|---------------------------------------| | N | 0 | - | 200 | - | | N | 10 | - | 200 | - | | Y | 0 | 200 | 200 | No segments deleted locally | | Y | 0 | 200 | 100 | Segments uploaded and deleted locally | | Y | 0 | 200 | 200 | All segments deleted locally | | Y | 10 | 10 | 200 | No segments deleted locally | | Y | 10 | 100 | 200 | Segments uploaded and deleted locally | | Y | 10 | 200 | 200 | All segments deleted locally | Reviewers: Kamal Chandraprakash <kamal.chandraprakash@gmail.com>
…anUnsubscribe to align test case (apache#20407) `testAsyncConsumerClassicConsumerSubscribeInvalidTopicCanUnsubscribe` does not align with the test case. This patch renames the test name to describe the test case more precisely. Reviewers: TengYao Chi <frankvicky@apache.org>
…call (apache#20375) This PR ensures that describeTopics correctly propagates its timeoutMs setting to the underlying describeCluster call. Integration tests were added to verify that the API now fails with a TimeoutException when brokers do not respond within the configured timeout. Reviewers: Ken Huang <s7133700@gmail.com>, TengYao Chi <kitingiao@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
…20395) The PR fixes the batch alignment issue when partitions are re-assigned. During initial read of state the batches can be broken arbitrarily. Say the start offset is 10 and cache contains [15-18] batch during initialization. When fetch happens at offset 10 and say the fetched batch contain 10 records i.e. [10-19] then correct batches will be created if maxFetchRecords is greater than 10. But if maxFetchRecords is less than 10 then last offset of batch is determined, which will be 19. Hence acquire method will incorrectly create a batch of [10-19] while [15-18] already exists. Below check is required t resolve the issue: ``` if (isInitialReadGapOffsetWindowActive() && lastAcquiredOffset > lastOffset) { lastAcquiredOffset = lastOffset; } ``` While testing with other cases, other issues were determined while updating the gap offset, acquire of records prior share partitions end offset and determining next fetch offset with compacted topics. All these issues can arise mainly during initial read window after partition re-assignment. Reviewers: Andrew Schofield <aschofield@confluent.io>, Abhinav Dixit <adixit@confluent.io>, Chirag Wadhwa <cwadhwa@confluent.io>
…s is empty (apache#20410) If `errorResults` is empty, there’s no need to create a new `entriesPerPartition` map. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
This PR removes associated logging within NetworkClient to reduce noise and streamline the client code. Reviewers: Ismael Juma <ismael@juma.me.uk>, David Arthur <mumrah@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
…nfig constants (apache#20249) This PR aims to add documentation to `alterLogLevelConfigs` method to remind users to use valid LogLevelConfig constants. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
…apache#16584) This is the first part of the implementation of [KIP-1023](https://cwiki.apache.org/confluence/display/KAFKA/KIP-1023%3A+Follower+fetch+from+tiered+offset) The purpose of this pull request is for the broker to start returning the correct offset when it receives a -6 as a timestamp in a ListOffsets API request. Added unit tests for the new timestamp. Reviewers: Kamal Chandraprakash <kamal.chandraprakash@gmail.com>
Adds example commands for running integration tests from the command line. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
apache#20405) - **Changes**: Replace misused dynamicPerBrokerConfigs with dynamicDefaultConfigs - **Reasons**: KRaft servers don't handle the cluser-level configs in starting from: https://github.com/apache/kafka/pull/18949/files#r2296809389 Reviewers: Jun Rao <junrao@gmail.com>, Jhen-Yung Hsu <jhenyunghsu@gmail.com>, PoAn Yang <payang@apache.org>, Chia-Ping Tsai <chia7712@gmail.com> --------- Co-authored-by: PoAn Yang <payang@apache.org>
As per the suggestion by @adixitconfluent and @chirag-wadhwa5, [here](apache#20395 (comment)), I have refactored the code with variable and method names. Reviewers: Andrew Schofield <aschofield@confluent.io>, Chirag Wadhwa <cwadhwa@confluent.io>
…ducer/consumer (apache#20390) As described in [jira](https://issues.apache.org/jira/browse/KAFKA-19625), this PR implements replace `consumer.config` and `producer.config` with `command-config` for kafka-verifiable-producer.sh and kafka-verifiable-consumer.sh. Reviewers: Andrew Schofield <aschofield@confluent.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.