Feat: Support client JWT auth method for kafka oidc #4057

acroca · 2025-10-06T16:39:24Z

At the moment the oidc auth type only supports authentication using clientID and clientSecret.

This PR adds a new auth type oidc_private_key_jwtthat uses certificates instead. The component will use those certificates to build the client assertion and call the IdP to get a token.

This new auth type uses the following new fields in the kafka components:

oidcClientAssertionCert
oidcClientAssertionKey
oidcResource
oidcAudience

In this PR I also added tests for both oidc authentication mechanisms against a local kafka, using keycloak as an IdP.

Signed-off-by: Albert Callarisa <[email protected]>

JoshVanL · 2025-10-10T14:59:33Z

common/component/kafka/metadata.go

+		if m.OidcClientAuthMethod == "client_secret" && m.OidcClientSecret == "" {
+			return nil, errors.New("kafka error: missing OIDC Client Secret for authType 'oidc' (client_secret)")
+		}
+		if m.OidcClientAuthMethod == "client_jwt" && m.OidcClientAssertionCert == "" {
+			return nil, errors.New("kafka error: missing OIDC Client Assertion Cert for authType 'oidc' (client_jwt)")
+		}
+		if m.OidcClientAuthMethod == "client_jwt" && m.OidcClientAssertionKey == "" {
+			return nil, errors.New("kafka error: missing OIDC Client Assertion Key for authType 'oidc' (client_jwt)")
 		}


Please can we use pointers to signal presence.

I used empty strings for consistence with the rest of the metadata fields. I think it'd be better to change it all in a different PR, wdyt?

JoshVanL · 2025-10-10T15:01:05Z

common/component/kafka/sasl_oauthbearer.go

+	block, _ := pem.Decode([]byte(ts.ClientAssertionKey))
+	if block == nil {
+		return nil, errors.New("invalid PEM private key for client assertion")
+	}
+	pk, err := x509.ParsePKCS8PrivateKey(block.Bytes)
+	if err != nil {
+		// try PKCS1
+		if rsaKey, err2 := x509.ParsePKCS1PrivateKey(block.Bytes); err2 == nil {
+			pk = rsaKey
+		} else {
+			return nil, fmt.Errorf("unable to parse private key: %w", err)
+		}
+	}
+	rsaKey, ok := pk.(*rsa.PrivateKey)
+	if !ok {
+		return nil, errors.New("client_jwt requires RSA private key")
+	}


Use dapr/kit/crypto/pem.

JoshVanL · 2025-10-10T15:01:27Z

common/component/kafka/sasl_oauthbearer.go

+		Subject(ts.ClientID).
+		Audience([]string{audClaim}).
+		IssuedAt(now).
+		Expiration(now.Add(1 * time.Minute)).


Why 1 minute?

It's arbitrary, it's just a short lived token used to communicate with the IdP. It could be even shorter in practice. Do you think we should make it shorter, like 5 seconds or so maybe?

pls ensure the docs reflect this as well when the time comes 🙏

JoshVanL · 2025-10-10T15:01:51Z

common/component/kafka/sasl_oauthbearer.go

+		urlValues.Set("scope", strings.Join(ts.Scopes, " "))
+	}
+
+	timeoutCtx, cancel := ctx.WithTimeout(ctx.TODO(), tokenRequestTimeout)


Why TODO?

It was TODO before too, I don't know the reason. Other places use the background context 🤷
The sarama.AccessTokenProvider(docs) doesn't take a context, wiring it up with a provided context would change a bit too much. Should we use background instead?

We should take a context which is passed through

JoshVanL · 2025-10-10T15:02:18Z

common/component/kafka/sasl_oauthbearer.go

+	defer cancel()
+	ts.configureClient()
+	req, err := http.NewRequestWithContext(timeoutCtx, http.MethodPost, ts.TokenEndpoint.TokenURL, strings.NewReader(urlValues.Encode()))
+	if err != nil {
+		return nil, err
+	}
+	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
+	resp, err := ts.httpClient.Do(req)
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
+		return nil, fmt.Errorf("token endpoint returned %d", resp.StatusCode)
+	}
+	var tr tokenResponse
+	if err := json.NewDecoder(resp.Body).Decode(&tr); err != nil {
+		return nil, err
+	}
+	if tr.AccessToken == "" {
+		return nil, errors.New("no access_token in response")
+	}
+	ts.CachedToken = oauth2.Token{AccessToken: tr.AccessToken, Expiry: time.Now().Add(time.Duration(tr.ExpiresIn) * time.Second)}
+	return ts.asSaramaToken(), nil


Is there not something that does all this for us in a library?

I couldn't find any unfortunately

JoshVanL · 2025-10-10T15:03:08Z

tests/certification/pubsub/kafka/components/auth_oidc_certs/kafka.yaml

+      -----BEGIN CERTIFICATE-----
+      MIIEJTCCAg2gAwIBAgIUS0RIkoWlMVd8UfE/I7IEzKD4kmwwDQYJKoZIhvcNAQEL
+      BQAwGDEWMBQGA1UEAwwNbG9jYWwtb2lkYy1jYTAeFw0yNTEwMDMxMDI3NDlaFw0y
+      ODAxMDYxMDI3NDlaMBsxGTAXBgNVBAMMEGRhcHItb2lkYy1jbGllbnQwggEiMA0G
+      CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDQIqGyGGlQRqXNDlSuHRDGmnJ4dQDb
+      jK/KxeC+wXQiDCSfHgP7NOjZOOCiQYKAUs9MRg+tKepxFRORZLAbRlU9kuU4Z7bg
+      E6wDMztCqTASUZtwZeN36iJDcQ++xnmAi8j0bvs7w00wrgPMXPnT09Sf9sQyEkR2
+      t26OLcQnZHwrUWHIbZeXqvRF7uYG00GKjFmqI+FjrmDx0hoENew+Jzpk7S/7m5t2
+      h/weVAr5REtGTIQxbX8nNmoO2JFbyILBcfP9M9R9zxiTEFVgP5sl2QfLFerPpULM
+      1tAiCO+oPk8CAFiD9TKe+HR/Us4uIxgRmxjgBnzmFSJjDflvK2DslpzPAgMBAAGj
+      ZDBiMAsGA1UdDwQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDAjAdBgNVHQ4EFgQU
+      oCgedGjEdPHOBRq6OwKKGUmpUXIwHwYDVR0jBBgwFoAUImUtm30c0E0Z1Xsep9o/
+      nXsqyGYwDQYJKoZIhvcNAQELBQADggIBAKafBCz1EUny2pKOI8xTHVkoZjhTvtm2
+      Ly2DicAEwtESXlBta64bOug4mTq8RLdKxLa8xSBSZxzygTfb+q8IcuuYL0/DAttZ


Please do not use hard coded certificates and keys in tests. They get copied and used in the real world, and they also expire.

Always generate them in the test, and use secret refs

Signed-off-by: Albert Callarisa <[email protected]>

bindings/kafka/metadata.yaml

Signed-off-by: Albert Callarisa <[email protected]>

tests/certification/pubsub/kafka/components/auth_oidc_certs/kafka.yaml

tests/certification/pubsub/kafka/data/realm-export.json

tests/certification/pubsub/kafka/docker-compose.auth.yml

tests/certification/pubsub/kafka/kafka_test.go

Signed-off-by: Albert Callarisa <[email protected]>

sicoyle · 2025-10-21T14:39:18Z

common/component/kafka/metadata.go

+		if m.OidcClientAssertionKey == "" {
+			return nil, errors.New("kafka error: missing OIDC Client Assertion Key for authType 'oidc_private_key_jwt'")
+		}
+		if m.OidcScopes != "" {


pls add this field to the auth profile in the metadata.yaml as I don't see it with the openid string default

sicoyle · 2025-10-21T14:40:25Z

common/component/kafka/metadata.go

+			k.logger.Warn("Warning: no OIDC scopes specified, using default 'openid' scope only. This is a security risk for token reuse.")
+			m.internalOidcScopes = []string{"openid"}
+		}
+		if m.OidcExtensions != "" {


i think this is also missing in the metadata.yaml file. So this is an optional field?

common/component/kafka/metadata_test.go

common/component/kafka/sasl_oauthbearer_private_key_jwt.go

Signed-off-by: Albert Callarisa <[email protected]>

Ref: dapr/components-contrib#4057 Signed-off-by: Albert Callarisa <[email protected]>

Feat: Support client assertion auth method for kafka oidc

9cd61ec

Signed-off-by: Albert Callarisa <[email protected]>

acroca force-pushed the client-assertion-oauth branch from c23e275 to 9cd61ec Compare October 6, 2025 17:02

acroca marked this pull request as ready for review October 9, 2025 07:40

acroca requested review from a team as code owners October 9, 2025 07:40

acroca force-pushed the client-assertion-oauth branch 2 times, most recently from 2e0fe18 to 6cb1166 Compare October 9, 2025 08:06

Added certification tests for both oidc auth mechanisms

742b291

Signed-off-by: Albert Callarisa <[email protected]>

acroca force-pushed the client-assertion-oauth branch from 6cb1166 to 742b291 Compare October 9, 2025 08:35

acroca changed the title ~~Feat: Support client assertion auth method for kafka oidc~~ Feat: Support client JWT auth method for kafka oidc Oct 9, 2025

JoshVanL requested changes Oct 10, 2025

View reviewed changes

acroca added 3 commits October 16, 2025 10:24

use dapr/kit to decode pem

3ec7932

Signed-off-by: Albert Callarisa <[email protected]>

Remove hardcoded certificates

b73242b

Signed-off-by: Albert Callarisa <[email protected]>

Merge branch 'main' into client-assertion-oauth

5a8677c

acroca requested a review from JoshVanL October 20, 2025 12:37

sicoyle reviewed Oct 20, 2025

View reviewed changes

bindings/kafka/metadata.yaml Outdated Show resolved Hide resolved

acroca force-pushed the client-assertion-oauth branch 3 times, most recently from 404144d to c3489f7 Compare October 21, 2025 09:52

acroca requested a review from sicoyle October 21, 2025 09:54

Created a new authType for oidc with private key jwt

8481041

Signed-off-by: Albert Callarisa <[email protected]>

acroca force-pushed the client-assertion-oauth branch from c3489f7 to 8481041 Compare October 21, 2025 10:19

JoshVanL previously approved these changes Oct 21, 2025

View reviewed changes

sicoyle reviewed Oct 21, 2025

View reviewed changes

Typo fixed

b34eb3e

Signed-off-by: Albert Callarisa <[email protected]>

acroca dismissed JoshVanL’s stale review via b34eb3e October 21, 2025 13:30

acroca requested review from JoshVanL and sicoyle October 21, 2025 13:30

sicoyle reviewed Oct 21, 2025

View reviewed changes

common/component/kafka/metadata_test.go Show resolved Hide resolved

common/component/kafka/sasl_oauthbearer_private_key_jwt.go Show resolved Hide resolved

Added scopes and extensions to the private key authentication metadata

d34cb70

Signed-off-by: Albert Callarisa <[email protected]>

sicoyle approved these changes Oct 21, 2025

View reviewed changes

JoshVanL approved these changes Oct 21, 2025

View reviewed changes

yaron2 merged commit c30af7d into dapr:main Oct 21, 2025
91 of 92 checks passed

This was referenced Oct 21, 2025

[1.16] Added oidc_private_key_jwt auth type #4064

Closed

[1.15] Added oidc_private_key_jwt auth type #4065

Closed

acroca deleted the client-assertion-oauth branch October 21, 2025 15:59

acroca mentioned this pull request Oct 21, 2025

[1.14] Added oidc_private_key_jwt auth type #4066

Closed

acroca added a commit to acroca/dapr-docs that referenced this pull request Oct 22, 2025

Docs for OAuth2 Private Key JWT

9e1c3d4

Ref: dapr/components-contrib#4057 Signed-off-by: Albert Callarisa <[email protected]>

This was referenced Oct 22, 2025

Docs for Kafka OAuth2 Private Key JWT dapr/docs#4922

Open

Add Support for certificate-based authentication in Kafka OIDC flow, #4003

Closed

Feat: Support client JWT auth method for kafka oidc #4057

Feat: Support client JWT auth method for kafka oidc #4057

Uh oh!

Conversation

acroca commented Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JoshVanL Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

acroca commented Oct 6, 2025 •

edited

Loading

JoshVanL Oct 21, 2025 •

edited

Loading