Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC-657 Add note about 30-day baked-in trial licenses for Redpanda 24.3 #853

Merged
merged 35 commits into from
Nov 24, 2024
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
2caf0c6
Update config for new beta
Deflaimun Oct 2, 2024
c3ffdd4
update with rpk 24.3.1-rc1 (#819)
Deflaimun Oct 21, 2024
58e0bff
DOC-502 License enforcement updates (#813)
JakeSCahill Oct 22, 2024
121d5f4
Properties 24 3 (#822)
Deflaimun Oct 22, 2024
19c0ec1
Add User resource docs (#773)
JakeSCahill Oct 23, 2024
fce336b
Leader pinning (#809)
kbatuigas Oct 28, 2024
3e93983
DOC-287 Mountable TS topics (#725)
kbatuigas Oct 30, 2024
03d3e60
Update with latest rpk commands from v0.0.0-20241104git4a0f859 (#835)
Deflaimun Nov 4, 2024
41bfe40
what's new in 24.3 beta (#811)
micheleRP Nov 4, 2024
afba214
Force-update fallback 24.3-rc2
Deflaimun Nov 4, 2024
488c9f6
Michele rp patch 1 (#837)
micheleRP Nov 6, 2024
67c90f5
DOC-470 Debug bundle in Redpanda Console (#825)
JakeSCahill Nov 12, 2024
16e9cda
add Tombstone property (#847)
Deflaimun Nov 12, 2024
58ded83
Tombstone retention (#829)
kbatuigas Nov 12, 2024
1b2ad07
Add note about 45-day baked-in trial licenses for Redpanda 24.3
JakeSCahill Nov 13, 2024
cd736ac
Apply suggestions from code review
JakeSCahill Nov 13, 2024
1e5f9de
Add Leader Pinning to enterprise list and address feedback
JakeSCahill Nov 13, 2024
ddbe343
Doc-476 - Backfill partitioning update (#842)
Feediver1 Nov 15, 2024
183d021
Doc 2599 - Configure Redpanda with a Customer-Managed Key (#808)
Feediver1 Nov 15, 2024
1b5614d
Update modules/get-started/pages/licensing/overview.adoc
JakeSCahill Nov 15, 2024
0cf4b28
DOC-759 fix link (#866)
micheleRP Nov 18, 2024
37f0a24
Merge branch 'v-WIP/24.3' into DOC-657
Feediver1 Nov 18, 2024
8457507
Apply suggestions from code review
JakeSCahill Nov 19, 2024
e5e936f
Merge branch 'v-WIP/24.3' into DOC-657
JakeSCahill Nov 19, 2024
7b441a1
Add note about upgrade restrictions
JakeSCahill Nov 20, 2024
c933dfb
More rpk examples and how to disable enterprise
JakeSCahill Nov 22, 2024
e8c7bc6
Merge branch 'v-WIP/24.3' into DOC-657
JakeSCahill Nov 22, 2024
4ced649
consistency
JakeSCahill Nov 22, 2024
775b73c
Apply suggestions from review
JakeSCahill Nov 23, 2024
126795c
Apply suggestions from review
JakeSCahill Nov 23, 2024
8b60146
Update monitor-license-status.adoc
JakeSCahill Nov 24, 2024
1d1fc55
Update monitor-license-status.adoc
JakeSCahill Nov 24, 2024
e7438cb
Update disable-enterprise-features.adoc
JakeSCahill Nov 24, 2024
7bef54d
Update whats-new.adoc
JakeSCahill Nov 24, 2024
f2b3ed6
Merge branch 'v-WIP/24.3' into DOC-657
JakeSCahill Nov 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 16 additions & 15 deletions modules/develop/pages/produce-data/leader-pinning.adoc
Original file line number Diff line number Diff line change
@@ -1,44 +1,45 @@
= Leader Pinning
// tag::single-source[]
:description: Learn about leader pinning and how to configure a preferred partition leader location based on cloud availability zones or regions.

Produce requests that write data to Redpanda topics go through the topic partition leader, which syncs messages across its follower replicas. For a Redpanda cluster deployed across multiple availability zones (AZs), leader pinning ensures that a topic's partition leaders are geographically closer to clients, which helps decrease networking costs and guarantees lower latency.

If consumers are located in the same preferred region or AZ for leader pinning, and you have not set up xref:develop:consume-data/follower-fetching.adoc[follower fetching], leader pinning can also help reduce networking costs on consume requests.

== Prerequisites
:description: Learn about Leader Pinning and how to configure a preferred partition leader location based on cloud availability zones or regions.

[NOTE]
====
include::shared:partial$enterprise-license.adoc[]
====

Before you can enable leader pinning, you must xref:manage:rack-awareness.adoc#configure-rack-awareness[configure rack awareness] on the cluster. If the config_ref:enable_rack_awareness,true,properties/cluster-properties[] cluster configuration property is set to `false`, leader pinning is disabled across the cluster.
Produce requests that write data to Redpanda topics go through the topic partition leader, which syncs messages across its follower replicas. For a Redpanda cluster deployed across multiple availability zones (AZs), Leader Pinning ensures that a topic's partition leaders are geographically closer to clients, which helps decrease networking costs and guarantees lower latency.

If consumers are located in the same preferred region or AZ for Leader Pinning, and you have not set up xref:develop:consume-data/follower-fetching.adoc[follower fetching], Leader Pinning can also help reduce networking costs on consume requests.

== Prerequisites

Before you can enable Leader Pinning, you must xref:manage:rack-awareness.adoc#configure-rack-awareness[configure rack awareness] on the cluster. If the config_ref:enable_rack_awareness,true,properties/cluster-properties[] cluster configuration property is set to `false`, Leader Pinning is disabled across the cluster.

== Configure Leader Pinning

== Configure leader pinning
You can use both a topic configuration property and a cluster configuration property to configure Leader Pinning.

You can use both a topic configuration property and a cluster configuration property to configure leader pinning.
You can set the topic configuration property for individual topics only, or set the cluster-wide configuration property that will enable Leader Pinning by default for all topics. You can also use a combination in which a default setting applies across the cluster, and you toggle the setting on or off for specific topics.

You can set the topic configuration property for individual topics only, or set the cluster-wide configuration property that will enable leader pinning by default for all topics. You can also use a combination in which a default setting applies across the cluster, and you toggle the setting on or off for specific topics.

This configuration is based on the following scenario: you have Redpanda deployed in a multi-AZ or multi-region cluster, and you have configured each broker so that the config_ref:rack,true,properties/broker-properties[] configuration property contains rack IDs corresponding to the AZ IDs:

* Set the topic configuration property xref:reference:properties/topic-properties.adoc#redpandaleaderspreference[`redpanda.leaders.preference`]. The property accepts the following string values:
+
--
** `none`: Opt out the topic from leader pinning.
** `none`: Opt out the topic from Leader Pinning.
** `racks:<rack1>[,<rack2>,...]`: Specify the preferred location (rack) of all topic partition leaders. The list can contain one or more rack IDs, and you can list the IDs in any order. Spaces in the list are ignored, for example: `racks:rack1,rack2` and `racks: rack1, rack2` are equivalent. You cannot specify empty rack IDs, for example: `racks: rack1,,rack2`. If you specify multiple IDs, Redpanda tries to distribute the partition leader locations equally across brokers in these racks.
--
+
This property inherits the default value from the cluster property `default_leaders_preference`.

* Set the cluster configuration property config_ref:default_leaders_preference,true,properties/cluster-properties[], which specifies the default leader pinning configuration for all topics that don’t have `redpanda.leaders.preference` explicitly set. It accepts values in the same format as `redpanda.leaders.preference`. Default: `none`
* Set the cluster configuration property config_ref:default_leaders_preference,true,properties/cluster-properties[], which specifies the default Leader Pinning configuration for all topics that don’t have `redpanda.leaders.preference` explicitly set. It accepts values in the same format as `redpanda.leaders.preference`. Default: `none`
+
This property also affects internal topics, such as `__consumer_offsets` and transaction coordinators. All offset tracking and transaction coordination requests get placed within the preferred regions or AZs for all clients, so you see end-to-end latency and networking cost benefits.

If there is more than one broker in the preferred AZ (or AZs), leader pinning distributes partition leaders uniformly across brokers in the AZ.
If there is more than one broker in the preferred AZ (or AZs), Leader Pinning distributes partition leaders uniformly across brokers in the AZ.

== Leader pinning failover across availability zones
== Leader Pinning failover across availability zones

If there are three AZs: A, B, and C, and A becomes unavailable, the failover behavior is as follows:

Expand Down
64 changes: 58 additions & 6 deletions modules/get-started/pages/licensing/overview.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -31,27 +31,79 @@ The Enterprise Edition is licensed with the https://github.com/redpanda-data/red

=== License keys

Enterprise features require an Enterprise Edition license key, sometimes referred to as Enterprise license, license key, or license. You can evaluate enterprise features with a free 30-day trial. To request a trial license key, to extend your trial period, or to purchase a license, contact https://redpanda.com/try-redpanda?section=enterprise-trial[Redpanda Sales^].
Enterprise features require an Enterprise Edition license key, sometimes referred to as Enterprise license, license key, or license.

Starting with version 24.3, new Redpanda clusters automatically receive a trial license that's valid for 30 days, allowing unrestricted use of Enterprise features. This evaluation period begins when the cluster is created for the first time. After this period expires, inactive Enterprise features are disabled, and active features enter a restricted state.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we expand on what is a restricted state?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's expanded on in the table. Restricted state means different things depending on the feature.


To request a trial license key, to extend your trial period, or to purchase a license, contact https://redpanda.com/try-redpanda?section=enterprise-trial[Redpanda Sales^].

[[self-managed]]
=== Enterprise features in Redpanda Self-Managed

The following enterprise features for Redpanda Self-Managed are activated with a valid Enterprise Edition license key:
The following table lists the available Enterprise features for Redpanda Self-Managed, along with the restrictions applied if the Enterprise license is invalid or has expired.

[cols="1,2,2"]
|===
| Feature | Description | Restrictions Without Valid License

| xref:manage:audit-logging.adoc[Audit Logging]
| Records detailed logs of cluster activities for compliance and monitoring.
| Read access to the audit log topic is denied, but logging continues.

| xref:manage:cluster-maintenance/continuous-data-balancing.adoc[Continuous Data Balancing]
| Automatically balances partitions across a cluster to optimize resource use and performance.
| Continuous balancing is disabled, reverting to the default `node_add` setting that balances partitions only after a broker is added to the cluster.

| xref:manage:cluster-maintenance/cluster-balancing.adoc#intra-broker-partition-balancing[Continuous Intra-Broker Partition Balancing (`core_balancing_continuous`)]
| Balances partition replicas across CPU cores in an individual broker to optimize disk space usage.
| Continuous Intra-Broker Partition Balancing is disabled.

| xref:manage:security/fips-compliance.adoc[FIPS Compliance]
| Enables compliance with FIPS security standards for cryptography.
| None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattschumpert Seems like...odd messaging for the 3 licensed features that have None for the consequence of not having a license. I suspect that is so we don't compromise the security of someone's cluster, but it also suggests that you can continue to use these 3 features without a license indefinitely. Is that the case?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Feediver1 correct. Our goal with the sanctions was to not cause users to lose access to the cluster (such as limiting kerberos or oidc authentication).


| xref:manage:security/authentication.adoc#kerberos[Kerberos authentication]
| Provides secure Kerberos-based authentication.
| None

include::get-started:partial$licensing/enterprise-features.adoc[tag=self-managed]
| xref:manage:security/authentication.adoc#kerberos[Leader Pinning]
| Specifies the set of availability zones where the leaders of partitions of a given topic should be located.
| Leader Pinning is disabled on all topics.

| xref:manage:security/authentication.adoc#oidc[OAUTHBEARER/OIDC authentication]
| Allows for OAUTHBEARER and OpenID Connect (OIDC) authentication.
| None

| xref:manage:remote-read-replicas.adoc[Remote Read Replicas]
| Enables remote clusters to read data stored in object storage for disaster recovery.
| Remote Read Replica topics cannot be created or modified.

| xref:manage:security/authorization/rbac.adoc[Role-Based Access Control (RBAC)]
| Manages user roles and permissions within the cluster.
| Roles and ACLs associated with roles cannot be created or modified. Role deletion is allowed.

| xref:manage:schema-reg/schema-id-validation.adoc[Server-Side Schema ID Validation]
| Validates schema IDs server-side to ensure schema compatibility. With schema ID validation, records associated with unregistered schemas are detected and dropped by a broker rather than a consumer.
| Topics with schema validation settings cannot be created or modified.

| xref:manage:tiered-storage.adoc[Tiered Storage]
| Enables storage of data in cloud object storage for long-term retention and retrieval.
| Topics cannot be created or modified to enable Tiered Storage features. Additional partitions cannot be added to topics that have any Tiered Storage properties enabled.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iceberg will be an enterprise feature too? Should we include it here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the feature is ready/documented 👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct - we decided against any sanctioning on this while it's in public beta

|===

[[console]]
=== Enterprise features in Redpanda Console

The following enterprise features for Redpanda Console are activated with a valid Enterprise Edition license key:

include::get-started:partial$licensing/enterprise-features.adoc[tag=console]
* xref:console:config/security/authentication.adoc[Redpanda Console Authentication]
* xref:console:config/security/authorization.adoc[Redpanda Console Authorization (RBAC)]

[[connect]]
=== Enterprise features in Redpanda Connect


include::get-started:partial$licensing/enterprise-features.adoc[tag=connect]
The Enterprise Edition of Redpanda Connect includes additional connectors. For a list of all available connectors, see xref:redpanda-connect:components:catalog.adoc[].

== How Redpanda Console handles licenses

Expand Down
27 changes: 14 additions & 13 deletions modules/manage/pages/cluster-maintenance/cluster-balancing.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,30 +10,31 @@ Cluster balancing is crucial for optimal performance. Unbalanced clusters can sa
| Topic-aware data balancer |Description

| <<Partition leadership balancing>>
| This balancer transfers the leadership of a broker's partitions to other replicas to avoid topic leadership hotspots on one or a few specific brokers in your cluster.
+
+
|
This balancer transfers the leadership of a broker's partitions to other replicas to avoid topic leadership hotspots on one or a few specific brokers in your cluster.
+
The partition leader regularly sends heartbeats to its followers. If a follower does not receive a heartbeat within a timeout, it triggers a new leader election. Redpanda also provides leadership balancing when brokers are added or decommissioned.

| <<Partition replica balancing>>
| This balancer moves partition replicas to avoid topic replica hotspots on one or a few specific brokers in your cluster.
+
+
|
This balancer moves partition replicas to avoid topic replica hotspots on one or a few specific brokers in your cluster.
+
Redpanda prioritizes balancing a topic's partition replica count evenly across all brokers while it's balancing the cluster's overall partition count. Because different topics in a cluster can have vastly different load profiles, this better distributes the workload evenly across brokers.
+
+

+
Redpanda provides partition replica balancing when brokers are added or decommissioned.

| <<Intra-broker partition balancing>>

| This balancer moves partition replicas across CPU cores in an individual broker. Redpanda maintains balanced partition replica assignments between cores to avoid topic hotspots on one or a few specific cores within a broker.
|
This balancer moves partition replicas across CPU cores in an individual broker. Redpanda maintains balanced partition replica assignments between cores to avoid topic hotspots on one or a few specific cores within a broker.
+
Continuous Intra-Broker Partition Balancing (`core_balancing_continuous`) requires an xref:get-started:licenses.adoc#redpanda-enterprise-edition[Enterprise license].

| xref:manage:cluster-maintenance/continuous-data-balancing.adoc[Continuous Data Balancing]

| This balancer monitors broker and rack availability, as well as disk usage, to avoid topic hotspots when moving data off brokers with fuller disks. Continuous Data Balancing enables self-healing clusters that dynamically balance partitions. It also ensures adherence to rack-aware replica placement policy and self-heals after rack (or availability zone) failure or replacement. This balancer does not keep the relative fullness of each broker within a defined range, it just prevents hitting the fullness threshold of each individual broker.
+
+
|
This balancer monitors broker and rack availability, as well as disk usage, to avoid topic hotspots when moving data off brokers with fuller disks. Continuous Data Balancing enables self-healing clusters that dynamically balance partitions. It also ensures adherence to rack-aware replica placement policy and self-heals after rack (or availability zone) failure or replacement. This balancer does not keep the relative fullness of each broker within a defined range, it just prevents hitting the fullness threshold of each individual broker.
+
Continuous Data Balancing requires an xref:get-started:licenses.adoc#redpanda-enterprise-edition[Enterprise license].
|===

Expand Down
4 changes: 2 additions & 2 deletions modules/reference/pages/properties/topic-properties.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -515,13 +515,13 @@ The preferred location (rack) for partition leaders of a topic.

This property inherits the value from the config_ref:default_leaders_preference,true,properties/cluster-properties[] cluster configuration property. You may override the cluster-wide setting by specifying the value for individual topics.

If the cluster configuration property config_ref:enable_rack_awareness,true,properties/cluster-properties[] is set to `false`, leader pinning is disabled across the cluster.
If the cluster configuration property config_ref:enable_rack_awareness,true,properties/cluster-properties[] is set to `false`, Leader Pinning is disabled across the cluster.

**Default**: `none`

**Values**:

- `none`: Opt out the topic from leader pinning.
- `none`: Opt out the topic from Leader Pinning.
- `racks:<rack1>[,<rack2>,...]`: Specify the preferred location (rack) of all topic partition leaders. The list can contain one or more rack IDs. If you specify multiple IDs, Redpanda tries to distribute the partition leader locations equally across brokers in these racks.

**Related topics**:
Expand Down
8 changes: 4 additions & 4 deletions modules/upgrade/pages/k-rolling-upgrade.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,12 @@ kubectl exec <pod-name> --namespace <namespace> -c redpanda -- \
[%collapsible]
====
The Redpanda version for each broker is listed under `BROKER-VERSION`.
[.no-copy]
[role="no-copy",subs="attributes+"]
```
NODE-ID BROKER-VERSION
0 v22.2.10
1 v22.2.10
2 v22.2.10
0 {latest-redpanda-tag}
1 {latest-redpanda-tag}
2 {latest-redpanda-tag}
```
====
- xref:upgrade:k-compatibility.adoc[Review the Kubernetes compatibility matrix] to find out if you need to upgrade the Helm chart or the Redpanda Operator to use your chosen version of Redpanda.
Expand Down