Skip to content

fix in parallel topic deletion and partition reassignment issue - #14

Merged
xiowu0 merged 2 commits into
linkedin:2.0-lifrom
xiowu0:reassign_deletion
Apr 25, 2019
Merged

fix in parallel topic deletion and partition reassignment issue#14
xiowu0 merged 2 commits into
linkedin:2.0-lifrom
xiowu0:reassign_deletion

Conversation

@xiowu0

@xiowu0 xiowu0 commented Apr 17, 2019

Copy link
Copy Markdown

kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignments of the deleted topic never complete because the topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller doesn't resume topic deletion for online broker and register broker notification handler (etc.) during onBrokerStartup.

The fix cleans up pending partition reassignment during topic deletion.

@xiowu0
xiowu0 requested review from hzxa21 and jjkoshy April 17, 2019 22:32
Comment thread core/src/main/scala/kafka/controller/TopicDeletionManager.scala Outdated
Comment thread core/src/test/scala/unit/kafka/admin/ReassignPartitionsClusterTest.scala Outdated

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this check necessary for this test?
Instead, could you just assertTrue(waitForReassignmentToComplete(0)) at the end?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to waitForReassignmentToComplete(). This is a void function check for partition reassignment znode, and also use default pause-time waitForReassignmentToComplete(pause: Long = 100L)

Comment thread core/src/main/scala/kafka/controller/KafkaController.scala Outdated
xiowu0 added 2 commits April 19, 2019 10:52
kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignment of the deleted topic never complete because the topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller doesn't resume topic deletion for online broker and register broker notification handler (etc.) during onBrokerStartup.

The fix cleans up pending partition reassignment during topic deletion.
@xiowu0
xiowu0 force-pushed the reassign_deletion branch from 98fb5de to 3fb2a8f Compare April 19, 2019 18:24

@hzxa21 hzxa21 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM to me. Thanks for the patch.

@xiowu0
xiowu0 merged commit dd29753 into linkedin:2.0-li Apr 25, 2019
xiowu0 added a commit that referenced this pull request Jun 11, 2019
… issue (#14)

TICKET = KAFKA-8249
LI_DESCRIPTION =

kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignment of the deleted topic never complete because the topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller doesn't resume topic deletion for online broker and register broker notification handler (etc.) during onBrokerStartup.

The fix cleans up pending partition reassignment during topic deletion.

EXIT_CRITERIA = TICKET [KAFKA-8249]
xiowu0 added a commit that referenced this pull request Jul 17, 2019
TICKET = KAFKA-8249
LI_DESCRIPTION =

kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignment of the deleted topic never complete because the topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller doesn't resume topic deletion for online broker and register broker notification handler (etc.) during onBrokerStartup.

The fix cleans up pending partition reassignment during topic deletion.

EXIT_CRITERIA = TICKET [KAFKA-8249]
xiowu0 added a commit that referenced this pull request Aug 22, 2019
TICKET = KAFKA-8249
LI_DESCRIPTION =

kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignment of the deleted topic never comes to completion after topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller not to resume topic deletion for online broker and not to register broker notification handler during onBrokerStartup.

The fix cleans up pending partition reassignment during topic deletion.

EXIT_CRITERIA = TICKET [KAFKA-8249]
xiowu0 added a commit that referenced this pull request May 29, 2020
TICKET = KAFKA-8249
LI_DESCRIPTION =

kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignment of the deleted topic never comes to completion after topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller not to resume topic deletion for online broker and not to register broker notification handler during onBrokerStartup.

The fix cleans up pending partition reassignment during topic deletion.

EXIT_CRITERIA = TICKET [KAFKA-8249]
gitlw pushed a commit that referenced this pull request Jun 12, 2020
TICKET = KAFKA-8249
LI_DESCRIPTION =

kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignment of the deleted topic never comes to completion after topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller not to resume topic deletion for online broker and not to register broker notification handler during onBrokerStartup.

The fix cleans up pending partition reassignment during topic deletion.

EXIT_CRITERIA = TICKET [KAFKA-8249]
gitlw pushed a commit that referenced this pull request Jun 13, 2020
TICKET = KAFKA-8249
LI_DESCRIPTION =

kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignment of the deleted topic never comes to completion after topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller not to resume topic deletion for online broker and not to register broker notification handler during onBrokerStartup.

The fix cleans up pending partition reassignment during topic deletion.

EXIT_CRITERIA = TICKET [KAFKA-8249]
lmr3796 pushed a commit that referenced this pull request Feb 9, 2022
TICKET = KAFKA-8249
LI_DESCRIPTION =

kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignment of the deleted topic never comes to completion after topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller not to resume topic deletion for online broker and not to register broker notification handler during onBrokerStartup.

The fix cleans up pending partition reassignment during topic deletion.

EXIT_CRITERIA = TICKET [KAFKA-8249]
lmr3796 pushed a commit that referenced this pull request Feb 9, 2022
TICKET = KAFKA-8249
LI_DESCRIPTION =

kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignment of the deleted topic never comes to completion after topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller not to resume topic deletion for online broker and not to register broker notification handler during onBrokerStartup.

The fix cleans up pending partition reassignment during topic deletion.

EXIT_CRITERIA = TICKET [KAFKA-8249]
lmr3796 pushed a commit that referenced this pull request Mar 25, 2022
TICKET = KAFKA-8249
LI_DESCRIPTION =

kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignment of the deleted topic never comes to completion after topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller not to resume topic deletion for online broker and not to register broker notification handler during onBrokerStartup.

The fix cleans up pending partition reassignment during topic deletion.

EXIT_CRITERIA = TICKET [KAFKA-8249]
lmr3796 pushed a commit that referenced this pull request Jun 2, 2022
TICKET = KAFKA-8249
LI_DESCRIPTION =

kafka allows topic deletion to complete successfully when there are pending partition reassignments of the same topics. This leads several issues: 1) pending partition reassignment of the deleted topic never comes to completion after topic is deleted. 2) onPartitionReassignment -> updateAssignedReplicasForPartition will throw out IllegalStateException for non-existing node. This in turns causes controller not to resume topic deletion for online broker and not to register broker notification handler during onBrokerStartup.

The fix cleans up pending partition reassignment during topic deletion.

EXIT_CRITERIA = TICKET [KAFKA-8249]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants