Add kerberos auth support to kudu connector by grantatspothero · Pull Request #10953 · trinodb/trino

grantatspothero · 2022-02-04T20:08:23Z

~~This PR is currently blocked by upgrading the kudu client version to 1.15.0 in this PR~~: https://github.com/trinodb/trino/pull/10940/files

This is because support for customizing the kerberos service principal primary name was only added in java client 1.15.0:
apache/kudu@e69b3c6

Documentation

( ) No documentation is needed.
(x) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

( ) No release notes entries required.
(x) Release notes entries required with the following suggested text:

# Kudu

* Add support for Kerberos authentication. ({issue}`10953`)

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KuduModule.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KuduAuthenticationType.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KuduModule.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KuduRecordCursor.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/schema/NoSchemaEmulation.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KuduClientConfig.java

...oduct-tests/src/main/java/io/trino/tests/product/kudu/TestKuduConnectoKerberosSmokeTest.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KerberizedKuduClient.java

...toolkit/src/main/java/io/trino/plugin/base/authentication/CachingKerberosAuthentication.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/IKuduClient.java

...plugin-toolkit/src/main/java/io/trino/plugin/base/authentication/KerberosAuthentication.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/DelegatingKuduClient.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KerberizedKuduClient.java

hashhar

Didn't review the kerberos config in the PT env - not an expert with it.

the only important question from me is regarding when to renew tickets and what happens if we hold on to an expired ticket.

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/DelegatingKuduClient.java

...toolkit/src/main/java/io/trino/plugin/base/authentication/CachingKerberosAuthentication.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KuduClientConfig.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KuduModule.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KuduSecurityModule.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/PassthroughKuduClient.java

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/SystemProperties.java

...roduct-tests-launcher/src/main/java/io/trino/tests/product/launcher/env/common/Kerberos.java

.../src/main/java/io/trino/tests/product/launcher/env/environment/EnvMultinodeKerberosKudu.java

...ts-launcher/src/main/java/io/trino/tests/product/launcher/suite/suites/Suite7NonGeneric.java

wendigo

Generally LGTM, some config names/descriptions improvements. Nice work @grantatspothero !

A subsequent commit will then introduce a kerberizedkuduclient that handles kerberos ticket renewal

Subsequent commit will add kerberos config support to the kudumodule

grantatspothero · 2022-02-16T17:44:05Z

Rebased and created docs. I also tried to take the suggestions on the config properties, let me know if the feedback I provided does not make sense.

hashhar · 2022-02-18T17:46:13Z

docs/src/main/sphinx/connector/kudu.rst

@mosabua Can you PTAL?

@grantatspothero are the newlines between consecutive properties needed?

~~No, just thought that made this more readable, can change.~~

@mosabua can you take a look

@hashhar the newlines were to match the existing style of the docs, see above in the same file

grantatspothero · 2022-02-18T18:50:32Z

Had to rebase this commit: fca8b7b

to fix the kudu product test CI failure. There was a race condition where we reported kerberos startup was successful before the kerberos init script finished running.

wendigo · 2022-02-21T10:45:35Z

See failures in suite-1: io.trino.tempto.query.QueryExecutionException: java.sql.SQLException: Query failed (#20220218_224931_02583_g3jkd): line 1:1: Catalog 'kudu' does not exist

...oduct-tests/src/main/java/io/trino/tests/product/kudu/TestKuduConnectoKerberosSmokeTest.java

wendigo · 2022-02-21T10:49:46Z

...oduct-tests/src/main/java/io/trino/tests/product/kudu/TestKuduConnectoKerberosSmokeTest.java

can we make it shorter? (configurable)? Can we expire it programmatically

I configured the default ticket expiry in tests to be 1 min, see kerberos_init.sh, I can make this lower just wanted to be very safe.

I'll look a bit more into useTicketCache in JAAS, it is possible we can force jaas to use the ticket cache then destroy the ticket cache programmatically
https://docs.oracle.com/javase/8/docs/jre/api/security/jaas/spec/com/sun/security/auth/module/Krb5LoginModule.html

When I tried to configure useTicketCache with debug logs on, it attempted to read a ticket cache file that did not exist.

It looks like the java kerberos client implementation does not ever write tickets to the ticket cache, it only supports reading from an existing ticket cache, see here for some more details:
https://stackoverflow.com/questions/45463635/jaas-fails-to-persist-kerberos-ticket-to-cache-file-and-unable-to-create-cach/45618313#45618313

Going down that path doesn't seem promising.

I'm not a kerberos expert, if other people have ideas let me know! Revocation of kerberos tickets is not usually done because ticket lifetimes are so short.

grantatspothero · 2022-02-22T23:43:59Z

@wendigo there were permissions problems with the kudu keytab files that only happened in CI for some reason, fixed and tests are green now.

wendigo

LGTM % modernizer violations:

Error:  /home/runner/work/trino/trino/plugin/trino-kudu/src/test/java/io/trino/plugin/kudu/TestKuduAuthenticationConfig.java:39: Use buildOrThrow() instead, as it makes it clear that it will throw on duplicated values
Error:  /home/runner/work/trino/trino/plugin/trino-kudu/src/test/java/io/trino/plugin/kudu/TestKuduKerberosConfig.java:55: Use buildOrThrow() instead, as it makes it clear that it will throw on duplicated values
Error:  /home/runner/work/trino/trino/plugin/trino-kudu/src/test/java/io/trino/plugin/kudu/TestKuduKerberosConfig.java:74: Use buildOrThrow() instead, as it makes it clear that it will throw on duplicated values

Includes handling of kerberos ticket expiration and product tests demonstrating ticket renewal

Allows reduction of boilerplate related to system properties

findepi · 2023-02-02T09:16:12Z

...oduct-tests/src/main/java/io/trino/tests/product/kudu/TestKuduConnectoKerberosSmokeTest.java

+            assertThat(result).updatedRowsCountIsEqualTo(25);
+            assertThat(onTrino().executeQuery(format("SELECT count(*) FROM %s", kuduTable))).containsExactlyInOrder(row(25));
+            // Kerberos tickets are configured to expire after 60 seconds, this should expire the ticket
+            Thread.sleep(70_000L);


We shouldn't have a test that sleeps 70 seconds, should we?

@findepi We could reduce the max ticket lifetime a bit, see MAX_TICKET_LIFETIME="1min" in kerberos init.

You cannot set this ticket lifetime too low or problems can occur. For example, setting the max ticket lifetime to 5s will cause the ticket to get refreshed repeatedly.

findepi · 2023-02-02T09:18:31Z

FYI, the product test is or started to be flaky -- #14441

cla-bot bot added the cla-signed label Feb 4, 2022

github-actions bot added the tests:hive label Feb 4, 2022

grantatspothero commented Feb 4, 2022

View reviewed changes

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KuduModule.java Outdated Show resolved Hide resolved

Praveen2112 reviewed Feb 8, 2022

View reviewed changes

grantatspothero commented Feb 8, 2022

View reviewed changes

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KuduClientConfig.java Outdated Show resolved Hide resolved

grantatspothero force-pushed the gn/kuduKerberosSupport branch 6 times, most recently from 433567c to d3bb26e Compare February 8, 2022 21:04

grantatspothero commented Feb 8, 2022

View reviewed changes

...oduct-tests/src/main/java/io/trino/tests/product/kudu/TestKuduConnectoKerberosSmokeTest.java Outdated Show resolved Hide resolved

grantatspothero commented Feb 8, 2022

View reviewed changes

plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/KerberizedKuduClient.java Outdated Show resolved Hide resolved

grantatspothero commented Feb 8, 2022

View reviewed changes

...toolkit/src/main/java/io/trino/plugin/base/authentication/CachingKerberosAuthentication.java Outdated Show resolved Hide resolved