Skip to content

Doc 1500 ephemeral auth #1210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jul 18, 2025
Merged

Doc 1500 ephemeral auth #1210

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
2a3ca40
docs: update ephemeral credentials usage in 25.2
Deflaimun Jul 15, 2025
0eb3704
update with guidance to activate security
Deflaimun Jul 15, 2025
f9a7883
remove note about ephemeral creds in SR
Deflaimun Jul 15, 2025
c198f6a
address review comment
Deflaimun Jul 15, 2025
2e0d279
Update modules/get-started/pages/release-notes/redpanda.adoc
paulohtb6 Jul 16, 2025
f3cfe14
Update modules/get-started/pages/release-notes/redpanda.adoc
paulohtb6 Jul 16, 2025
31edeb2
adjust recommendation
Deflaimun Jul 16, 2025
e5a07f1
review comments
Jul 17, 2025
2092749
review comments
Jul 17, 2025
0e40080
fix rolling upgrade
Jul 17, 2025
f019a52
split message
paulohtb6 Jul 17, 2025
fdb8a2c
relative links
paulohtb6 Jul 17, 2025
cb4a1e7
adjust key and message
paulohtb6 Jul 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions modules/get-started/pages/release-notes/redpanda.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,29 @@ The Redpanda Admin API now includes new health probes to help you ensure safe br
* xref:manage:cluster-maintenance/rolling-restart.adoc[]
* xref:upgrade:rolling-upgrade.adoc[]

== HTTP Proxy authentication changes

[IMPORTANT]
====
**Breaking change**: Starting in Redpanda 25.2, ephemeral credentials for HTTP Proxy are removed. If your HTTP Proxy API listeners use `authentication_method: none`, you must configure explicit SASL credentials for HTTP Proxy to authenticate with the Kafka API.
====

HTTP Proxy previously used automatically generated ephemeral credentials to authenticate with the Kafka API when the HTTP Proxy listeners had `authentication_method: none`. These ephemeral credentials are now removed to improve security and simplify the authentication model.

Without explicit SASL credentials configured, HTTP Proxy attempts unauthenticated connections to the Kafka API.

Redpanda Data recommends configuring authentication for HTTP Proxy to ensure secure communication with the Kafka API.

**Action required**: Configure the following HTTP Proxy client properties in your `redpanda.yaml` configuration:

- xref:reference:properties/broker-properties.adoc#scram_username[`scram_username`]: Username for SASL/SCRAM authentication
- xref:reference:properties/broker-properties.adoc#scram_password[`scram_password`]: Password for SASL/SCRAM authentication
- xref:reference:properties/broker-properties.adoc#sasl_mechanism[`sasl_mechanism`]: SASL mechanism (typically `SCRAM-SHA-256` or `SCRAM-SHA-512`)

For detailed configuration instructions, see xref:manage:security/authentication.adoc#schema-and-http-to-redpanda[Configure HTTP Proxy to connect to Redpanda with SASL].

**Note**: This change does not affect Schema Registry, which continues to use ephemeral credentials.

== Redpanda Console v3.0.0

The Redpanda Console v3.0.0 release includes the following updates:
Expand Down
33 changes: 31 additions & 2 deletions modules/manage/partials/authentication.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -657,10 +657,19 @@ endif::[]

Schema Registry and HTTP Proxy connect to Redpanda over the Kafka API. For the Kafka username and password, Redpanda uses ephemeral credentials internal to the cluster. Ephemeral credentials are regular SCRAM credentials, but they're only stored in memory and are lost when a broker restarts. When the Schema Registry or HTTP Proxy start up, they broadcast an ephemeral credential to other brokers over the internal RPC. If authentication fails to a particular broker, new ephemeral credentials are sent to that broker, and the service reconnects.

[IMPORTANT]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am checking this part with the Kubernetes team to see what we currently do.

Either way, we'll need to update the paragraph before this section because we will no longer be using ephemeral credentials.

cc @chrisseto

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oooooooooh, dear. This is probably going to break some things... AFAIK we set authentication_method: none by default. We really don't like to generate passwords on behalf of users as it's causes a lot of trouble.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should really be enabling auth on pandaproxy and kafka API together. The fact that most people don't use pandaproxy and just leave it alone probably means there are a bunch of clusters that have root access to their cluster exposed via pandaproxy.

break some things...

Yeah, but it's for better security and TBH I really suspect very few people actually use the HTTP proxy for anything.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We really don't like to generate passwords on behalf of users as it's causes a lot of trouble.

TBH the right fix is not to generate passwords IMO, but to enable auth for HTTP proxy.

====
*Breaking change in Redpanda 25.2:* Ephemeral credentials for HTTP Proxy are removed. If your HTTP Proxy API listeners use `authentication_method: none`, you must configure explicit SASL credentials for HTTP Proxy to authenticate with the Kafka API. Without these credentials, HTTP Proxy attempts unauthenticated connections.

Redpanda Data recommends configuring authentication for HTTP Proxy to ensure secure communication with the Kafka API.

This change does not affect Schema Registry, which continues to use ephemeral credentials.
====

Schema Registry and HTTP Proxy support only the SASL/SCRAM mechanism.

ifndef::env-kubernetes[]
You can override the ephemeral credentials and manually configure Schema Registry and HTTP Proxy to connect to Redpanda with SASL. This approach is particularly useful when you want to apply specific access control through ACLs for the Schema Registry and HTTP Proxy users.
Starting in Redpanda 25.2, you must manually configure HTTP Proxy to connect to Redpanda with SASL when HTTP Proxy API listeners use `authentication_method: none`. For Schema Registry, you can override the ephemeral credentials and manually configure Schema Registry to connect to Redpanda with SASL. This approach is particularly useful when you want to apply specific access control through ACLs for the Schema Registry and HTTP Proxy users.

. xref:manage:security/authorization/index.adoc[Create appropriate ACLs] for the Schema Registry and HTTP Proxy users to define and restrict their access rights within the Redpanda cluster.

Expand Down Expand Up @@ -698,7 +707,22 @@ schema_registry_client:
sasl_mechanism: SCRAM-SHA-256
----
+
For HTTP Proxy:
For HTTP Proxy (required when `authentication_method: none` starting in Redpanda 25.2):
+
[,yaml]
----
pandaproxy_client:
brokers:
- address: 127.0.0.1
port: 9092
scram_username: <username>
scram_password: <password>
sasl_mechanism: SCRAM-SHA-256
----
+
When HTTP Proxy API listeners use `authentication_method: none`, the HTTP Proxy client uses these credentials to authenticate with the Kafka API. The user specified in `scram_username` must have appropriate permissions to access the required Kafka resources.
+
If TLS is enabled for the Kafka API, additional configuration is required:
+
[,yaml]
----
Expand Down Expand Up @@ -1757,6 +1781,11 @@ For all available endpoints, see xref:api:ROOT:pandaproxy-schema-registry.adoc[]

To disable authentication for a listener, set `authentication_method` to `none`:

[IMPORTANT]
====
*Breaking change in Redpanda 25.2:* When HTTP Proxy API listeners use `authentication_method: none`, you must configure explicit SASL credentials for HTTP Proxy to authenticate with the Kafka API. Configure the `pandaproxy_client` properties: `scram_username`, `scram_password`, and `sasl_mechanism`. Without these credentials, HTTP Proxy will attempt unauthenticated connections to the Kafka API. Redpanda Data recommends configuring authentication for HTTP Proxy. For more information, see xref:reference:properties/broker-properties.adoc#scram_username[HTTP Proxy Client properties].
====

ifdef::env-kubernetes[]
[tabs]
======
Expand Down
14 changes: 11 additions & 3 deletions modules/reference/pages/properties/broker-properties.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -778,7 +778,11 @@ Delay (in milliseconds) for initial retry backoff.

=== sasl_mechanism

The SASL mechanism to use when connecting.
The SASL mechanism to use when the HTTP Proxy client connects to the Kafka API. These credentials are used when the HTTP Proxy API listener has `authentication_method: none` but the cluster requires authenticated access to the Kafka API.

Starting in Redpanda 25.2, ephemeral credentials for HTTP Proxy are removed. If your HTTP Proxy listeners use `authentication_method: none`, you must configure these SASL properties for HTTP Proxy to authenticate with the Kafka API.

For more information, see xref:manage:security/authentication.adoc#schema-and-http-to-redpanda[Configure HTTP Proxy to connect to Redpanda with SASL].

*Visibility:* `user`

Expand All @@ -790,7 +794,9 @@ The SASL mechanism to use when connecting.

=== scram_password

Password to use for SCRAM authentication mechanisms.
Password to use for SCRAM authentication mechanisms when the HTTP Proxy client connects to the Kafka API. This property is required when the HTTP Proxy API listener has `authentication_method: none` but the cluster requires authenticated access to the Kafka API.

Starting in Redpanda 25.2, ephemeral credentials for HTTP Proxy are removed. You must configure this property if your HTTP Proxy listeners use `authentication_method: none`.

*Visibility:* `user`

Expand All @@ -802,7 +808,9 @@ Password to use for SCRAM authentication mechanisms.

=== scram_username

Username to use for SCRAM authentication mechanisms.
Username to use for SCRAM authentication mechanisms when the HTTP Proxy client connects to the Kafka API. This property is required when the HTTP Proxy API listener has `authentication_method: none` but the cluster requires authenticated access to the Kafka API.

Starting in Redpanda 25.2, ephemeral credentials for HTTP Proxy are removed. You must configure this property if your HTTP Proxy listeners use `authentication_method: none`.

*Visibility:* `user`

Expand Down
2 changes: 2 additions & 0 deletions modules/upgrade/partials/incompat-changes.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
=== Review incompatible changes

* Starting in version 25.2, ephemeral credentials for HTTP Proxy are removed. If your HTTP Proxy API listeners use `authentication_method: none`, you must configure explicit SASL credentials (xref:reference:properties/broker-properties.adoc#scram_username[`scram_username`], xref:reference:properties/broker-properties.adoc#scram_password[`scram_password`], and xref:reference:properties/broker-properties.adoc#sasl_mechanism[`sasl_mechanism`]) for HTTP Proxy to authenticate with the Kafka API. Without these credentials, HTTP Proxy will attempt unauthenticated connections to the Kafka API. Redpanda Data recommends configuring authentication for HTTP Proxy. For more information, see xref:manage:security/authentication.adoc#schema-and-http-to-redpanda[Configure HTTP Proxy to connect to Redpanda with SASL].

* Redpanda Console v3.0.0 introduces breaking changes. If you are using Redpanda Console v2.x, xref:migrate:console-v3.adoc[review the migration guide] to address breaking changes before upgrading Redpanda Console.

* Starting in version 24.2, when managing configuration properties using the AlterConfigs API directly, Redpanda resets all unspecified values to the default values. This aligns more closely with the behavior in Apache Kafka. There is no change if you're managing your configuration with tools like `rpk`, Redpanda Console, Kubernetes, Helm, or Terraform.
Expand Down