Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MINOR; Fix incompatible change to the kafka config #16464

Merged
merged 2 commits into from
Jun 28, 2024

Conversation

jsancio
Copy link
Member

@jsancio jsancio commented Jun 26, 2024

Prior to KIP-853, users were not allow to enumerate listeners specified in controller.listener.names in the advertised.listeners. This decision was made in 3.3 because the controller.quorum.voters property is in effect the list of advertised listeners for all of the controllers.

KIP-853 is moving away from controller.quorum.voters in favor of a dynamic set of voters. This means that the user needs to have a way of specifying the advertised listeners for controller.

This change allows the users to specify listener names in controller.listener.names in advertised.listeners. To make this change forwards compatible (use a valid configuration from 3.8 in 3.9), the controller's advertised listeners are going to get computed by looking up the endpoint in advertised.listeners. If it doesn't exist, the controller will look up the endpoint in the listeners configuration.

This change also includes a fix the to the BeginQuorumEpoch request where the default value for VoterId was 0 instead of -1.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@jsancio jsancio added kraft KIP-853 KRaft Controller Membership Changes labels Jun 26, 2024
@jsancio jsancio marked this pull request as ready for review June 27, 2024 00:31
@jsancio jsancio changed the title MINOR; Fix incomaptible change to the kafka config MINOR; Fix incompatible change to the kafka config Jun 27, 2024
Copy link
Contributor

@cmccabe cmccabe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Next time we are going to have mark the new RPC versions as unstable, though.

@jsancio jsancio merged commit 9be27e7 into apache:trunk Jun 28, 2024
1 check failed
@jsancio jsancio deleted the minor-fix-advertise-controller-listener branch June 28, 2024 01:24
omkreddy added a commit to confluentinc/kafka that referenced this pull request Jun 28, 2024
abhi-ksolves pushed a commit to ksolves/kafka that referenced this pull request Jul 31, 2024
Prior to KIP-853, users were not allow to enumerate listeners specified in `controller.listener.names` in the `advertised.listeners`. This decision was made in 3.3 because the `controller.quorum.voters` property is in effect the list of advertised listeners for all of the controllers.

KIP-853 is moving away from `controller.quorum.voters` in favor of a dynamic set of voters. This means that the user needs to have a way of specifying the advertised listeners for controller.

This change allows the users to specify listener names in `controller.listener.names` in `advertised.listeners`. To make this change forwards compatible (use a valid configuration from 3.8 in 3.9), the controller's advertised listeners are going to get computed by looking up the endpoint in `advertised.listeners`. If it doesn't exist, the controller will look up the endpoint in the `listeners` configuration.

This change also includes a fix the to the BeginQuorumEpoch request where the default value for VoterId was 0 instead of -1.

Reviewers: Colin P. McCabe <[email protected]>
@emasab
Copy link
Contributor

emasab commented Oct 22, 2024

Warning: the fallback works only if the CONTROLLER listener host is routable, e.g. not for 0.0.0.0

emasab added a commit to confluentinc/librdkafka that referenced this pull request Dec 13, 2024
Mock handler implementation
Rename current consumer protocol from generic to classic
Mock handler with automatic or manual assignment
More consumer group metadata getters
Test helpers
Configurable session timeout and HB interval
Fix mock handler ListOffsets response
LeaderEpoch instead of CurrentLeaderEpoch
Integration tests passing with AK trunk
Improve documentation and KIP 848 specific mock tests
Add mock tests for unknown topic id in metadata request and partial reconciliation
Make test 0147 more reliable
Fix test 0106 after HB timeout change
Exclude test case with AK trunk
Rename rd_kafka_buf_write_tags to rd_kafka_buf_write_tags_empty
Trivup 0.12.5 can run a KafkaCluster directly with KRaft and AK trunk
Trivup 0.12.6 build with a specific commit
Trivup 0.12.7 with fixes for AK 3.8.0 and Py 3.12
New version of trivup 0.12.7 to fix an issue with apache/kafka#16464 on AK > 3.8.0
Static group membership mock tests
Move test 0147 to a different PR
Disable interactive "needsrestart" prompt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
KIP-853 KRaft Controller Membership Changes kraft
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants