Change status of intra broker balancing from beta to GA #855
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for redpanda-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
4 tasks
kbatuigas
requested review from
ztlpn,
mattschumpert,
daisukebe and
wzzzrd86
|
@daisukebe @wzzzrd86 added you as reviewers so you can eyeball and compare with #850, thanks! |
ztlpn
reviewed
Nov 13, 2024
| In Redpanda, every partition replica is assigned to a CPU core on a broker. While Redpanda's default <<partition-replica-balancing,partition balancing>> monitors cluster-level events, such as the addition of new brokers or broker failure to balance partition assignments, it does not account for the distribution of partitions _within_ an individual broker. | ||
|
|
||
| Prior to Redpanda version 24.2, this meant that some cores on a broker could inadvertently host many partitions of heavily-used topics and cause the CPU to be xref:manage:monitoring.adoc#cpu-usage[overburdened]. Additionally, when the partition rebalance moved some partitions away from a broker, the remaining partitions did not necessarily rebalance across the broker's cores. Or, if a broker's core count was increased, Redpanda did not assign any partitions to the new cores until new partitions were created or old partitions were moved out. | ||
|
|
||
| Starting in v24.2, topic-aware intra-broker partition balancing allows for dynamically reassigning partitions within a broker. Redpanda prioritizes an even distribution of a topic's partition replicas across all cores in a broker. If a broker's core count changes, when the broker starts back up, Redpanda can check partition assignments across the broker's cores and reassign partitions, so that a balanced assignment is maintained across all cores. Redpanda can also check partition assignments when partitions are added to or removed from a broker, and rebalance the remaining partitions between cores. | ||
|
|
||
| NOTE: Decreasing the number of CPU cores in a running cluster is supported from v24.2 only. |
There was a problem hiding this comment.
Will 24.2 docs still contain info about the curl command? Otherwise there is no way for the users to enable it for 24.2
There was a problem hiding this comment.
yes 24.2 is currently what's in main. Before we release 24.3, we cut a new maintenance branch off main for 24.2 and then merge v-WIP/24.3 into main.
There was a problem hiding this comment.
I think we can remove this note. Add the ability to decrease core count in production to the ‘Whats new’
There was a problem hiding this comment.
I think we can remove this note. Add the ability to decrease core count in production to the ‘Whats new’
Agreed. End users might not realize that the ability to decrease core count is tied to intra-broker partition balancing. Additionally, this is a significant improvement and deserves a mention in the 'What's New' section.
There was a problem hiding this comment.
Thank you @JakeSCahill @daisukebe , we'll add a note in our upcoming What's New: https://github.com/redpanda-data/docs/pull/865/files#diff-7e9028daec2320182888e36f1be6d5a941c79362fab72135786f5b0cb0456a30R20
JakeSCahill
reviewed
Nov 15, 2024
| curl -X PUT -d '{"state": "active"}' http://127.0.0.1:9644/v1/features/node_local_core_assignment | ||
| ``` | ||
| ==== | ||
|
|
||
| In Redpanda, every partition replica is assigned to a CPU core on a broker. While Redpanda's default <<partition-replica-balancing,partition balancing>> monitors cluster-level events, such as the addition of new brokers or broker failure to balance partition assignments, it does not account for the distribution of partitions _within_ an individual broker. | ||
|
|
||
| Prior to Redpanda version 24.2, this meant that some cores on a broker could inadvertently host many partitions of heavily-used topics and cause the CPU to be xref:manage:monitoring.adoc#cpu-usage[overburdened]. Additionally, when the partition rebalance moved some partitions away from a broker, the remaining partitions did not necessarily rebalance across the broker's cores. Or, if a broker's core count was increased, Redpanda did not assign any partitions to the new cores until new partitions were created or old partitions were moved out. |
There was a problem hiding this comment.
I think we should remove this. Anything prior to 24.2 is irrelevant for 24.3 users.
JakeSCahill
reviewed
Nov 15, 2024
| curl -X PUT -d '{"state": "active"}' http://127.0.0.1:9644/v1/features/node_local_core_assignment | ||
| ``` | ||
| ==== | ||
|
|
||
| In Redpanda, every partition replica is assigned to a CPU core on a broker. While Redpanda's default <<partition-replica-balancing,partition balancing>> monitors cluster-level events, such as the addition of new brokers or broker failure to balance partition assignments, it does not account for the distribution of partitions _within_ an individual broker. | ||
|
|
||
| Prior to Redpanda version 24.2, this meant that some cores on a broker could inadvertently host many partitions of heavily-used topics and cause the CPU to be xref:manage:monitoring.adoc#cpu-usage[overburdened]. Additionally, when the partition rebalance moved some partitions away from a broker, the remaining partitions did not necessarily rebalance across the broker's cores. Or, if a broker's core count was increased, Redpanda did not assign any partitions to the new cores until new partitions were created or old partitions were moved out. | ||
|
|
||
| Starting in v24.2, topic-aware intra-broker partition balancing allows for dynamically reassigning partitions within a broker. Redpanda prioritizes an even distribution of a topic's partition replicas across all cores in a broker. If a broker's core count changes, when the broker starts back up, Redpanda can check partition assignments across the broker's cores and reassign partitions, so that a balanced assignment is maintained across all cores. Redpanda can also check partition assignments when partitions are added to or removed from a broker, and rebalance the remaining partitions between cores. |
There was a problem hiding this comment.
Again, referencing the version in versioned docs is distracting.
4 tasks
ztlpn
approved these changes
Nov 18, 2024
kbatuigas
deleted the
DOC-673-Change-status-of-intra-broker-balancing-from-beta-to-GA
branch
Deflaimun
pushed a commit
that referenced
this pull request
Nov 19, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
node_local_core_assignmentfeature flag. The flag is now enabled by default as of 24.3.node_local_core_assignmentis enabled - the same feature flag that enables intra-broker partition balancing.Resolves https://github.com/redpanda-data/documentation-private/issues/
Review deadline: 15 Nov
Page previews
24.3 > Cluster balancing > Intra-broker partition balancing
Checks