KAFKA-2092: New partitioning for better load balancing#69
Closed
gdfm wants to merge 3 commits into
Closed
Conversation
Author
|
Passes gradle test |
|
kafka-trunk-git-pr #156 SUCCESS |
ymatsuda
pushed a commit
to ymatsuda/kafka
that referenced
this pull request
Sep 23, 2015
Close the iterator at the end in ProcessorJob.
efeg
pushed a commit
to efeg/kafka
that referenced
this pull request
Jan 29, 2020
…tion (apache#69) We are having more and more goals now. We are holding the lock for too long. This patch releases the lock between each goal to avoid holding it for all the goals.
Contributor
|
Adding new partitioning logic will require a KIP. The process is documented here: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals. I am going to close this PR for now. |
smccauliff
pushed a commit
to smccauliff/kafka
that referenced
this pull request
Oct 8, 2020
… issue (apache#69) TICKET = LI_DESCRIPTION = DynamicBrokerConfig holds a reference to DynamicConfig.Broker.ClusterLevelConfigs. During DynamicConfig object initialization, DynamicBrokerConfig object is created if it was not created before. kafka-configs.sh script (ConfigCommand.scala) tries to create DynamicConfig object directly, which in turns creates DynamicBrokerConfig object. This fix makes sure that when initializing DynamicBrokerConfig object, DynamicConfig.Broker.ClusterLevelConfigs should have already been defined. EXIT_CRITERIA = MANUAL [""]
wyuka
pushed a commit
to wyuka/kafka
that referenced
this pull request
Jan 19, 2022
This also squashes the directly related test commits - e59b0da [LI-HOTFIX] Add test case for describe maintenance broker config (apache#248) - 1f7bab6 [LI-HOTFIX] fix DynamicConfig and DynamicBrokerConfig cross reference issue (apache#69) TICKET = KAFKA-8527 LI_DESCRIPTION = When a broker is masked as maintenance broker, Kafka will not assign partitions of new topics to the broker if partition assignment is done by kafka brokers. 1) add dynamic broker config to support maintenance brokers 2) add logic to update maintenance broker info in controller 3) support maintenance broker in Kafka API 4) support maintenance broker in AdminZkClient 5) add logic to rearrange replica assignment for new topics for maintenance brokers 5) test cases RB=1484649 G=Kafka-Code-Reviews R=agencer, amendhek, jkoshy, zahuang A=zahuang EXIT_CRITERIA = TICKET [KAFKA-8527]
wyuka
pushed a commit
to wyuka/kafka
that referenced
this pull request
Mar 4, 2022
This also squashes the directly related test commits - e59b0da [LI-HOTFIX] Add test case for describe maintenance broker config (apache#248) - 1f7bab6 [LI-HOTFIX] fix DynamicConfig and DynamicBrokerConfig cross reference issue (apache#69) TICKET = KAFKA-8527 LI_DESCRIPTION = When a broker is masked as maintenance broker, Kafka will not assign partitions of new topics to the broker if partition assignment is done by kafka brokers. 1) add dynamic broker config to support maintenance brokers 2) add logic to update maintenance broker info in controller 3) support maintenance broker in Kafka API 4) support maintenance broker in AdminZkClient 5) add logic to rearrange replica assignment for new topics for maintenance brokers 5) test cases RB=1484649 G=Kafka-Code-Reviews R=agencer, amendhek, jkoshy, zahuang A=zahuang EXIT_CRITERIA = TICKET [KAFKA-8527]
wyuka
pushed a commit
to wyuka/kafka
that referenced
this pull request
Mar 28, 2022
This also squashes the directly related test commits - e59b0da [LI-HOTFIX] Add test case for describe maintenance broker config (apache#248) - 1f7bab6 [LI-HOTFIX] fix DynamicConfig and DynamicBrokerConfig cross reference issue (apache#69) TICKET = KAFKA-8527 LI_DESCRIPTION = When a broker is masked as maintenance broker, Kafka will not assign partitions of new topics to the broker if partition assignment is done by kafka brokers. 1) add dynamic broker config to support maintenance brokers 2) add logic to update maintenance broker info in controller 3) support maintenance broker in Kafka API 4) support maintenance broker in AdminZkClient 5) add logic to rearrange replica assignment for new topics for maintenance brokers 5) test cases RB=1484649 G=Kafka-Code-Reviews R=agencer, amendhek, jkoshy, zahuang A=zahuang EXIT_CRITERIA = TICKET [KAFKA-8527]
wyuka
pushed a commit
to wyuka/kafka
that referenced
this pull request
Jun 16, 2022
This also squashes the directly related test commits - e59b0da [LI-HOTFIX] Add test case for describe maintenance broker config (apache#248) - 1f7bab6 [LI-HOTFIX] fix DynamicConfig and DynamicBrokerConfig cross reference issue (apache#69) TICKET = KAFKA-8527 LI_DESCRIPTION = When a broker is masked as maintenance broker, Kafka will not assign partitions of new topics to the broker if partition assignment is done by kafka brokers. 1) add dynamic broker config to support maintenance brokers 2) add logic to update maintenance broker info in controller 3) support maintenance broker in Kafka API 4) support maintenance broker in AdminZkClient 5) add logic to rearrange replica assignment for new topics for maintenance brokers 5) test cases RB=1484649 G=Kafka-Code-Reviews R=agencer, amendhek, jkoshy, zahuang A=zahuang EXIT_CRITERIA = TICKET [KAFKA-8527]
rustd
pushed a commit
to rustd/pranavfinaldemokafka
that referenced
this pull request
Feb 9, 2024
…he#12743) (apache#69) The original code path paused the main consumer for all tasks before entering the restoration section of the code, and then resumed all after restoration has finished. In the new state updater part of the code, tasks that do not require restoration skip the restoration completely. They remain with the TaskManger and are never transferred to the StateUpdater, and thus are never resumed. This change makes sure that tasks that remain with the TaskManager are not paused. Reviewers: Guozhang Wang <wangguoz@gmail.com>, Bruno Cadonna <cadonna@apache.org> Co-authored-by: Lucas Brutschy <lucasbru@users.noreply.github.com>
davide-armand
pushed a commit
to aiven/kafka
that referenced
this pull request
Dec 1, 2025
fvaleri
pushed a commit
to fvaleri/kafka
that referenced
this pull request
Feb 2, 2026
Allow old version of kafka to be the source cluster
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.