[NEW VERSION] Trino 438

[maltesander]

Which new version of Trino should we support?

438 (or better 439 if released)

Our latest supported is 428.

Additional information

Changelog https://trino.io/docs/current/release/release-438.html

Changes required

436 (required)

• Require JDK 21.0.1 to run Trino Require JDK 21 to run Trino trinodb/trino#20010

434 (check for usage in iceberg)

• Disallow invalid configuration options. Previously, they were silently ignored Remove HiveConfig binding from Delta and Iceberg with Glue  trinodb/trino#19735
• Remove support for legacy table statistics tracking Remove support for pre-Puffin Iceberg stats trinodb/trino#19803

Notes

I just built 438 as custom image in experimental and there were a couple of things to do, just to keep it here:

docker-images

• use default storage connector (no custom image for 438)
• remove opa authorizer from docker image (for 438, need switch for older versions)
• add java 21 to trino for 438 and to java-base

trino-operator

• set opa.allow-permission-management-operations=true for views / grants in opa config (access-control properties, need switch for version 428/438)
• adapt JVM properties https://trino.io/docs/current/installation/deployment.html#jvm-config (need switch for version 428/438 because of different jdk version)

remarks:

• opa config / access-control properties can not be overridden / properties added
• JVM configs hardcoded can only be overridden completly

Implementation checklist

Please don't change anything in this list.
Not all of these steps are necessary for all versions.

• Update the Docker image
• Update documentation to include supported version(s)
• Update and test getting started guide with updated version(s)
• Update operator to support the new version (if needed)
• Update integration tests to test use the new versions (in addition or replacing old versions
• Update examples to use new versions

[paulpaul1076]

When do you plan to release this? They fixed the slow execution of "select count(*)" in trino 430, I am looking forward to this release.

[maltesander]

Hi @paulpaul1076, we have a meeting next monday on which product versions to include. Will update here then.

[maltesander]

Hi @paulpaul1076 we will introduce Trino version 436 in the Stackable 24.3 release line. The image maybe available earlier in the nightly / 0.0.0-dev version.

[paulpaul1076]

@maltesander thanks! When do you plan to make this release? Soon, or in several months?

[maltesander]

@paulpaul1076 the next release is planned for march, the 436 version is probably available as a 0.0.0-dev version before that (since we have to implement and test). Cannot be more specific sorry.

[soenkeliebau]

We should try to go to 437 directly, as this has been released in the interim.

I have copied the release notes below, seems to me like there are no relevant changes for us in there, the breaking changes are not in any functionality our operator uses.

Release Notes - https://trino.io/docs/current/release/release-437.html

General#
Add support for char(n) values in to_utf8(). (#20158)

Add support for char(n) values in lpad(). (#16907)

⚠️ Replace the exchange.compression-enabled configuration property and exchange_compression session property with theexchange.compression-codecand exchange_compression_codec properties, respectively. (#20274)

⚠️ Replace the spill-compression-enabled configuration property with the spill-compression-codec property. (#20274)

⚠️ Remove the deprecated experimental.spill-compression-enabled configuration property. (#20274)

Fix materialized views being permanently stale when they reference table functions. (#19904)

Fix failure when invoking functions that may return null values. (#18456)

Fix ArrayIndexOutOfBoundsException with RowBlockBuilder during output operations. (#20426)

Delta Lake connector#
Improve query performance for queries that don’t use table statistics. (#20054)

Hive connector#
Fix error when coercing union-typed data to a single type when reading Avro files. (#20310)

Iceberg connector#
Improve performance of queries with filters on ROW columns stored in Parquet files. (#17133)

[maltesander]

@soenkeliebau as discussed today it will be at least 438 for OPA? I can adapt title etc then...

[soenkeliebau]

I think you can, yes. As far as I can see 438 should be released later today..

[maltesander]

Ok will wait for release notes and update after.

[maltesander]

Hi @paulpaul1076,

we now have Trino 438 in our nexus https://repo.stackable.tech/#browse/browse:docker:v2%2Fstackable%2Ftrino%2Ftags%2F438-stackable0.0.0-dev

Due to a Java bump and some authorizer changes you have to use the 0.0.0-dev operator to run this. It will be available and stable in the next release coming in march 2024.

Malte

[paulpaul1076]

@maltesander Thanks for the update!

[paulpaul1076]

@maltesander what do you think about waiting a little bit and including the version of Trino that supports Iceberg views?

As you may know, Iceberg is the native table format for Trino. And they're about to release a new important feature: Iceberg views, here's the pull request:

trinodb/trino#19818

According to Daniel Weeks from Tabular and the comments in that PR, they're just waiting for Iceberg 1.5.0 release to go through (the vote is happening right now and it should be out this week).

From slack: