Add Trino 411 release notes

[colebow]

Description

Assemble the release notes for the upcoming Trino 411 release. Fixes #16453

Release notes

(x) This is not user-visible or docs only and no release notes are required.

Verification for each pull request

Format: maintainer, PR/issue number, ✅ / ❌ rn ✅ / ❌ docs
✅ rn - release note added and verified, or assessed to be not necessary, set to ❌ rn before completion
✅ docs - need for docs assessed and merged, or assessed to be not necessary, set to ❌ docs before completion

8 Mar 2023

• Support inserting into Delta Lake table having invariants #16136 ✅ rn ✅ docs
• Use air.test.jvm.additional-arguments in Ignite #16449 ✅ rn ✅ docs

9 Mar 2023

• Add ConnectorSession to TableFunctionProcessorProvider#getSplitProcessor #16443 ✅ rn ✅ docs
• Use deterministic map in equality inference #16440 ✅ rn ✅ docs
• Support predicate pushdown with like in Ignite #16398 ✅ rn ❌ docs
• Remove deprecated Elasticsearch passthrough query  #13390 ✅ rn ✅ docs
• Use lambda #16467 ✅ rn ✅ docs
• Use air.test.jvm.additional-arguments in BigQuery #16463 ✅ rn ✅ docs
• Add CI infrastructure for trino-redshift #16142 ✅ rn ✅ docs
• Support Iceberg materialized views with types not supported directly in Iceberg library #16050 ❌ rn ✅ docs
• Update action-surefire-report #16471 ✅ rn ✅ docs

10 Mar 2023

• Throw TableNotFoundException when SQLServer error is 208 #16487 ✅ rn ✅ docs
• Fix variable name #16493 ✅ rn ✅ docs
• Add coercion test for unpartitioned tables in hive #16119 ✅ rn ✅ docs
• Fix test-other-modules failure due to lack of Redshift secrets #16484 ✅ rn ✅ docs
• Extract meaningful methods in join operator #16468 ✅ rn ✅ docs
• Update action-surefire-report to fix processing large report files #16498 ✅ rn ✅ docs
• Store operator summaries in MysqlEventListener #16288 ✅ rn ✅ docs
• Avoid CI log spam when downloading dependencies #16505 ✅ rn ✅ docs
• Fix generation of JSON path keyvalue() IDs #16482 ✅ rn ✅ docs
• Update to Airbase 134 #16420 ✅ rn ✅ docs

11 Mar 2023

• Avoid CI log spam when downloading dependencies #16506 ✅ rn ✅ docs
• Remove bogus overload in tests #16490 ✅ rn ✅ docs
• Don't enforce top-level repo name in BaseCostBasedPlanTest #16486 ✅ rn ✅ docs
• Implement Materialized View GRACE PERIOD #15842 ✅ rn ✅ docs

12 Mar 2023

• Correct misuse of Description annotation in Config #16513 ✅ rn ✅ docs

13 Mar 2023

• Correct misuse of Description annotation in Config #16513 ✅ rn ✅ docs
• Deserialize partition value by physical name for Delta connector #16388 ✅ rn ✅ docs
• Bump redshift-jdbc42 to 2.1.0.12 #16022 ✅ rn ✅ docs
• Add test for SharedResource #16509 ✅ rn ✅ docs
• Cancel background Glue partition futures on failure #16418 ✅ rn ✅ docs
• Enable colocated join by default #16381 ✅ rn ✅ docs
• Add spilled data size to QueryStatistics #16442 ✅ rn ✅ docs
• Support Ignite (inner, left, right) join pushdown #16429 ✅ rn ❌ docs
• Graceful Task Termination #15478 ✅ rn ❌ docs
• Add regex to Hive table properties format row #16530 ✅ rn ✅ docs
• Fix overly restrictive correctness checks in native line readers #16531 ✅ rn ✅ docs
• reorder azure storage #16353 ✅ rn ✅ docs
• Add Hive tests for trailing space in location #16407 ✅ rn ✅ docs
• Add support for migrate procedure in Iceberg #13493 ✅ rn ✅ docs

14 Mar 2023

• Allow ViewExpression use session user explicitly #16436 ❌ rn ❌ docs
• Expose JMX RMI in product tests #16517 ✅ rn ✅ docs
• Fix FilesTable for non-null key_metadata, split_offsets, equality_ids #16519 ✅ rn ✅ docs
• Add sinh function to return hyperbolic sine #16535 ✅ rn ✅ docs
• Impersonation string edits #16529 ✅ rn ✅ docs
• Log impacted features #16526 ✅ rn ✅ docs
• Fix BaseFailureRecoveryTest temp table checking flakiness #16277 #16499 ✅ rn ✅ docs
• Fix Pinot Timeboundary handling #16511 ✅ rn ✅ docs
• Get stats from hive metastore only for the necessary columns  #16203 ✅ rn ✅ docs
• Improve error message for catalog errors when listing metadata #16497 ✅ rn ✅ docs
• Add native OpenX JSON format #16073 ✅ rn ❌ docs
• Update airlift units to 1.8 #14406 ✅ rn ✅ docs
• Fix InterruptedException in miscellaneous connectors #16550 ✅ rn ✅ docs
• Deprecate MISSING_DATA error codes and cleanup error handling #16556 ✅ rn ✅ docs
• Remove Hadoop usages from Iceberg #16454 ✅ rn ✅ docs
• Upgrade jsonwebtoken to 0.11.5 #16548 ✅ rn ✅ docs

15 Mar 2023

• Minor code cleanup #16527 ✅ rn ✅ docs
• Fix validation of session variable MAX_HASH_PARTITION_COUNT #16563 ✅ rn ✅ docs
• Fix spelling mistake #16560 ✅ rn ✅ docs
• Close its source exchanges once a stage finishes execution #16446 ✅ rn ✅ docs
• Update to Jersey 2.39 #16401 ✅ rn ✅ docs
• Support Iceberg refs system table #15649 ✅ rn ❌ docs
• Do not inline non-deterministic expressions as projection #16518 ✅ rn ✅ docs
• Remove unused methods from DeltaLakeTableHandle #16537 ✅ rn ✅ docs
• Restore skip archive handler in Iceberg Glue catalog #16564 ✅ rn ✅ docs

16 Mar 2023

• Add REFS to TestIcebergGlueCatalogAccessOperations.testSelectSystemTable #16576 ✅ rn ✅ docs
• Fix stage scheduling logic in FTE mode #16586 ✅ rn ✅ docs
• Remove Iceberg connector tests for REST and JDBC catalogs #16588 ✅ rn ✅ docs
• Avoid loading lineitem unnecessarily #16587 ✅ rn ✅ docs
• Include checked arguments in checks messages #16594 ✅ rn ✅ docs
• Improve docs for skewness function #16554 ✅ rn ✅ docs
• Add clean up for DeltaLake ANALYZE tests #16590 ✅ rn ✅ docs
• Simplify predicates involving year function #16106 ✅ rn ✅ docs
• Fix failure when query table function contains column alias in JDBC connectors #16235 ✅ rn ✅ docs
• Add example for TRINO_PASSWORD usage #16390 ✅ rn ✅ docs

17 Mar 2023

• Avoid deserialization when no columns are needed #16595 ✅ rn ✅ docs
• Add upper limit on number of tasks that take part in writing stages #16238 ✅ rn ✅ docs
• Update documentation for table functions #16432 ✅ rn ✅ docs
• Print diagnostic messages to stderr when testing docker image #16610 ✅ rn ✅ docs
• Fix hashing in BlackHoleNodePartitioningProvider #16599 ✅ rn ✅ docs
• Prevent Hive from returning table handles for Hudi tables #16605 ❌ rn ✅ docs
• Avoid extra allocation in OrderByOperator#getOutput() #16447 ✅ rn ✅ docs
• Allow to disable transactional directory listing cache #16614 ✅ rn ✅ docs
• Remove language codes from external URLs #16314 ✅ rn ✅ docs
• Update Hive catalog example name documentation #16477 ✅ rn ✅ docs
• Replace the deprecated method getCause and getRootCause with cause rootCause respectively #16606 ✅ rn ✅ docs
• Eagerly discard FileSingleStreamSpiller encryption keys #16307 ✅ rn ✅ docs

18 Mar 2023

• Fix merge doc written demo value error #16578 ✅ rn ✅ docs
• Set maxOutputLength correctly #16604 ✅ rn ✅ docs
• Add FixedSplitSource constructors #16360 ✅ rn ✅ docs

19 Mar 2023

• Make hive.metastore-stats-cache-ttl >= to hive.metastore-cache-ttl #16625 ✅ rn ❌ docs
• Improve LongSumAggregation and DoubleSumAggregation serde #16624 ✅ rn ✅ docs
• Use existing variable for file length #16448 ✅ rn ✅ docs
• Remove unused method from QueryManagerStats #16601 ✅ rn ✅ docs
• Replace usage of hadoop.fs.Path with String in Delta Lake #16256 ✅ rn ✅ docs
• Add statistics collection for added column #16109 ✅ rn ✅ docs
• [trino-ignite] Ensure primary_key columns exist while creating table #16266 ✅ rn ✅ docs
• Add a metadata table showing commits in Hudi connector #16181 ✅ rn ✅ docs

20 Mar 2023

• Simplify column adaptations for parquet in iceberg #16547 ✅ rn ✅ docs
• Refactor PredicatePushdown test with simpler query #16461 ✅ rn ✅ docs
• Minor code cleanup in Iceberg connector #16607 ✅ rn ✅ docs
• Add generated expression to extra info in Delta Lake #16631 ✅ rn ✅ docs
• Avoid boxing in DeltaLakeBucketFunction #16635 ✅ rn ✅ docs
• Fix BaseFailureRecoveryTest temp table checking message pattern #16636 ✅ rn ✅ docs
• Update Iceberg table statistics on inserts #15441 ✅ rn ✅ docs
• Add test for identity column in Delta Lake connector #16632 ✅ rn ✅ docs

21 Mar 2023

• Add hive.metastore-stats-cache-ttl doc #16642 ✅ rn ✅ docs
• Support coercing char types of different lengths in hive #16402 ✅ rn ✅ docs
• Fix incorrect result when MongoDB query table function contains helper functions #16626 ✅ rn ✅ docs
• Add link to duration prop type in JDBC config fragment #16638 ✅ rn ✅ docs
• Fix BigQuery tests for column names with special characters #16613 ✅ rn ✅ docs
• Add version requirement for Hudi connector #16246 ✅ rn ✅ docs

22 Mar 2023

• Disable testReadMetadataWithRelationsConcurrentModifications on H2 because of flakes #16659 ✅ rn ✅ docs
• Refactor TestQueryRunnerUtil #16648 ✅ rn ✅ docs
• Fix test class name in BigQuery cloud-tests profile #16666 ✅ rn ✅ docs
• BaseJdbcClient and FTE test cleanup #16379 ✅ rn ✅ docs
• Use JDBC parameters in JDBC complex expression pushdown #16616 ❌ rn ✅ docs
• Flush transaction log cache in Delta flush_metadata_cache procedure #16466 ✅ rn ✅ docs
• Remove extra digest copy during Digest merge #16668 ✅ rn ✅ docs
• Reduce number of TDigestHistogram allocations #16669 ✅ rn ✅ docs
• Tune JDBC fetch-size automatically based on column count #16644 ✅ rn ✅ docs
• Remove redundant LongDoubleState interface #16672 ✅ rn ✅ docs
• Fix import of wrong Preconditions class #16627 ✅ rn ✅ docs
• Test Iceberg cost-based plans with small files on TPC-DS #16645 ✅ rn ✅ docs
• Upgrade to Pinot 0.12.1 #16623 ✅ rn ✅ docs
• Simplify MaxDataSizeForStats and SumDataSizeForStats #16675 ✅ rn ✅ docs

23 Mar 2023

• Skip listing Glue tables with invalid column types #16677 ✅ rn ✅ docs
• Exclude all synthetic columns in applyProjection validation #16687 ✅ rn ✅ docs
• Remove Iceberg, Delta $data system table #16650 ✅ rn ✅ docs
• Docker build with Ubuntu mirrors #16098 ✅ rn ✅ docs
• Add retries in TestImpersonation #16694 ✅ rn ✅ docs
• Decrease number of old JDBC drivers tested #16684 ✅ rn ✅ docs
• Test parquet column mapping using name based and index based mapping #16698 ✅ rn ✅ docs
• Change t_char of AbstractTestHiveFileFormats from partition key to general column #16688 ✅ rn ✅ docs

24 Mar 2023

• Set thread name for DDL queries #16686 ✅ rn ✅ docs
• Introduce new methods projected & exceptColumns take string varargs in QueryAssertions #16609 ✅ rn ✅ docs
• Refactor Delta's DROP TABLE support for corrupted tables #16651 ✅ rn ✅ docs
• Remove unnecessary override method in phoenix metadata #16706 ✅ rn ✅ docs
• Make queryModifier final #16708 ✅ rn ✅ docs
• Refactor testRenameTableToLongTableName to remove Iceberg override #16707 ✅ rn ✅ docs
• Fix TestIcebergInsert.testIcebergConcurrentInsert timeout on CI #16692 ✅ rn ✅ docs
• Reduce lock contention on coordinator #16585 ✅ rn ✅ docs
• Make setup of bucketed tables in HiveQueryRunner optional #16705 ✅ rn ✅ docs
• Remove superfluous accumulator add function #16711 ✅ rn ✅ docs
• Fix failure when partition column contains uppercase in Iceberg #16713 ✅ rn ✅ docs
• Remove duplicate getParquetType method from ParquetPageSourceFactory #16699 ✅ rn ✅ docs
• Remove redundant boolean state from LongDecimalWithOverflowAndLongState #16667 ✅ rn ✅ docs
• Reduce synchronization on PipelinedStageExecution #16717 ✅ rn ✅ docs

25 Mar 2023

• Add NullablePosition to SumDataSizeForStats #16726 ✅ rn ✅ docs

26 Mar 2023

• Remove unused HiveHudiPartitionInfo.getTable method #14652 ✅ rn ✅ docs

27 Mar 2023

• Support DELETE statement in Ignite connector #16720 ✅ rn ✅ docs
• Add an example JDBC connector plugin #15004 ✅ rn ✅ docs
• Fix typo #16740 ✅ rn ✅ docs
• Split createPartitionedDriverRunner for partitioned and upartitioned … #16608 ✅ rn ✅ docs
• More string format removals #15316 ✅ rn ✅ docs
• Fix bad parameter count in code generator when column uses double slots #15979 ✅ rn ✅ docs
• Add max-outstanding-splits-size to Hive configuration doc #16367 ✅ rn ✅ docs
• Add missing groups to TestIcebergProcedureCalls.testMigrateHiveBucketedOnMultipleColumns #16731 ✅ rn ✅ docs
• Remove unused updatedColumns from IcebergTableHandle #16678 ✅ rn ✅ docs
• Remove OkHttp as a runtime dependency for the engine #16629 ✅ rn ✅ docs
• Update to ASM 9.4 #16628 ✅ rn ✅ docs
• Update object storage definition in glossary #16615 ✅ rn ✅ docs
• Minor cleanups in aggregation functions #16751 ✅ rn ✅ docs
• Fix Kerberos ticket refresh #16680 ✅ rn ✅ docs

28 Mar 2023

• Extend the list of status codes retried in client #16491 ✅ rn ✅ docs
• Allow listening for single container events #16752 ✅ rn ✅ docs
• fix typo in hive parquet doc #16767 ✅ rn ✅ docs
• Add test for trailing space in location in hive metadata #16544 ✅ rn ✅ docs
• Remove unnecessary and brittle tests #16765 ✅ rn ✅ docs
• Upgrade Confluent version to 7.3.1 #16763 ✅ rn ✅ docs

29 Mar 2023

• Explicitly set input dir in job conf instead of FileInputFormat.setInputPath which makes an IO call #16640 ✅ rn ✅ docs
• Improve the performance of filter functions #16681 ✅ rn ✅ docs
• Add quantile_at_value function #16736 ✅ rn ✅ docs
• Use parent partitioning for aggregations #16328 ✅ rn ✅ docs
• Add table function to execute stored procedure in SQLServer #16696 ✅ rn ✅ docs
• Use URI path for Glue location in tests #16771 ✅ rn ✅ docs
• Test trino-main with JDK 20 #16676 ✅ rn ✅ docs
• Fix handling of various table types in BigQuery connector #16737 ✅ rn ✅ docs
• Allow Iceberg MV with partitioning transforms on timestamptz #16637 ✅ rn ✅ docs

[findepi]

@colebow can you please update?

[martint]

What's a "generated expression"?

[colebow]

I believe it's generated column expressions?
cc @ebyhr