Skip to content

KAFKA-17613: Remove ZK migration code#17293

Merged
cmccabe merged 5 commits into
apache:trunkfrom
cmccabe:KAFKA-17613
Oct 3, 2024
Merged

KAFKA-17613: Remove ZK migration code#17293
cmccabe merged 5 commits into
apache:trunkfrom
cmccabe:KAFKA-17613

Conversation

@cmccabe

@cmccabe cmccabe commented Sep 26, 2024

Copy link
Copy Markdown
Contributor

Remove the controller machinery for doing ZK migration in Kafka 4.0.

As a note, I've taken the approach of leaving KafkaConfig.migrationEnabled but hard-coding it to
false here. That was because I didn't want to mess around with KafkaServer.scala and related classes.
Since they're getting removed soon anyway, it would be a waste.

@mumrah mumrah left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see anything obvious, but this might collide with #17284

@cmccabe

cmccabe commented Sep 26, 2024

Copy link
Copy Markdown
Contributor Author

As a note, I've taken the approach of leaving KafkaConfig.migrationEnabled but hard-coding it to false here. That was because I didn't want to mess around with KafkaServer.scala and related classes. Since they're getting removed soon anyway, it would be a waste.

@mumrah

mumrah commented Sep 27, 2024

Copy link
Copy Markdown
Member

There are some test failures. I think they'll be fixed after we merge #17284

@mumrah

mumrah commented Sep 27, 2024

Copy link
Copy Markdown
Member

@cmccabe the other removal PR is merged now, can you merge in trunk?

@chia7712 chia7712 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cmccabe thanks for this cleanup.

@@ -76,8 +70,6 @@ public class QuorumControllerMetrics implements AutoCloseable {
private final AtomicLong dualWriteOffset = new AtomicLong(0);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is dual-write metrics still used in production code after KRaftMigrationDriver is removed?

@cmccabe cmccabe Sep 30, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, I will remove this.

setFinalizedFeatures(currentFeatures.finalizedFeatures()).
setFinalizedFeaturesEpoch(currentFeatures.finalizedFeaturesEpoch()).
setZkMigrationEnabled(zkMigrationEnabled).
setZkMigrationEnabled(false).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not removing the setZkMigrationEnabled?

@cmccabe cmccabe Sep 30, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We cannot remove zkMigrationEnabled from the RPC without breaking backwards compatibility. Although it does default to false, so we could leave out this line (which I think might be more confusing to read, but would be technically correct).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can hardcode zkMigrationReady to false when building ApiVersionsResponse (ApiVersionsResponse#Builder#build) and then remove this setter ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't that make testing harder? It seems fine to just set it here.

@cmccabe

cmccabe commented Oct 1, 2024

Copy link
Copy Markdown
Contributor Author

I've updated this to remove the lingering reference to dual write offset in the metrics class.

[edit: also merged in trunk as @mumrah suggested. Thanks for the reviews, everyone.]

@mumrah mumrah left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chia7712

chia7712 commented Oct 2, 2024

Copy link
Copy Markdown
Member

I believe those failing tests can be removed if they are specifically designed to run with ZooKeeper migration

@chia7712

chia7712 commented Oct 3, 2024

Copy link
Copy Markdown
Member

@cmccabe Could you please fix the conflicts? the failed tests were removed by #17318

@cmccabe
cmccabe merged commit 85bfdf4 into apache:trunk Oct 3, 2024
@cmccabe
cmccabe deleted the KAFKA-17613 branch October 3, 2024 19:01
tedyu pushed a commit to tedyu/kafka that referenced this pull request Jan 6, 2025
Remove the controller machinery for doing ZK migration in Kafka 4.0.

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, David Arthur <mumrah@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants