Add release notes for 0.272

[asjadsyed]

Missing Release Notes

Amit Dutta

• Throw proper error code when timezone is invalid from packDateTimeWit… #17363 Throw proper error code when timezone is invalid from packDateTimeWit… (Merged by: Timothy Meehan)
• 3913a1a Enable Retrying max requests queued exception.

George Wang

• Add TPCDS schema to HiveQueryRunner #17132 Add TPCDS schema to HiveQueryRunner (Merged by: Ying)

Ke Wang

• Add streaming aggregation #17281 Add streaming aggregation (Merged by: James Sun)

Sergey Smirnov

• Thriftify ServerInfo #17391 Thriftify ServerInfo (Merged by: Timothy Meehan)

Tim Meehan

• Add adaptive execution policy #17378 Add adaptive execution policy (Merged by: Timothy Meehan)
• 290bc4a Fix buffer size calculation in TDigest

Todd Gao

• Add lark sheets connector #17218 Add lark sheets connector (Merged by: Beinan)

Extracted Release Notes

• Add a new JDBC driver parameter - timeZoneID #16756 (Author: v-jizhang): Add a new JDBC driver parameter - timeZoneID
  • Add a new JDBC driver parameter - timeZoneID :doc:/installation/jdbc :issue:16680.
• Support iceberg connector concurrent insertion #16983 (Author: panyliu): Support iceberg connector concurrent insertion
  • Support concurrent insertion from the same presto cluster or mutiple presto clusters which sharing the same metastore.
• Ensure Parameters are in proper order for queries having WITH clause #17083 (Author: v-jizhang): Ensure Parameters are in proper order for queries having WITH clause
  • Ensure Parameters are in proper order for queries having WITH clause.
• Improve listing performance of Hudi tables #17244 (Author: Sagar Sumit): Improve listing performance of Hudi tables
  • Upgrade Hudi version to 0.10.1.
  • Support metadata-based listing and bootstrap for Hudi tables.
• Skip reading Parquet pages using Column Indexes feature. #17284 (Author: Xinli shang): Skip reading Parquet pages using Column Indexes feature.
  • Add support to use Parquet page-level statistics to skip pages.
  • Introduce a flag "hive.parquet-column-index-filter-enabled" to turn on/off this feature. By default it is off.
• Enable cost based optimization by default #17288 (Author: Rongrong Zhong): Enable cost based optimization by default
  • Enable cost based optimization by default.
• Fix parquet data page offset calculation in parquet writer #17295 (Author: v-jizhang): Fix parquet data page offset calculation in parquet writer
  • Fix parquet data page offset calculation in parquet writer.
• [Pinot Connector] Support querying Pinot JSON type #17323 (Author: Xiang Fu): [Pinot Connector] Support querying Pinot JSON type
  • Support querying Pinot JSON type.
• Bump Alluxio client from 2.7.0 to 2.7.3 #17327 (Author: David N Perkins): Bump Alluxio client from 2.7.0 to 2.7.3
  • Bumped Alluxio client from 2.7.0 to 2.7.3.
• Spelling #17393 (Author: Josh Soref): Spelling
  • Fix the spelling of the write concern option JOURNAL_SAFE for the property mongodb.write-concern.
• Add secure_random functions #17399 (Author: Darren Fu): Add secure_random functions
  • Add secure_random() and secure_random(lower, upper) with uniform distribution.

All Commits

• 7cb1fef Bump up sphinx version to fix docs build (James Sun)
• 290bc4a Fix buffer size calculation in TDigest (Tim Meehan)
• 3913a1a Enable Retrying max requests queued exception. (Amit Dutta)
• 8479f26 Fix presto on spark tests being skipped (Ariel Weisberg)
• e7f1401 Kill queries aggressively under high task pressure (Mayank Garg)
• 1e64daf Expose ORC writer compression level as session property (Sergii Druzkin)
• 028ef8e Fix TestPrestoQueries (Ariel Weisberg)
• 8750895 Add lark sheets connector (Todd Gao)
• 36cfca4 Support isOrderable trait for distinct types (Pranjal Shankhdhar)
• 3c800f0 Add tests for DictionaryBlock size methods (James Petty)
• 2a79ba1 Make DictionaryBlock sizeInBytes calculation consistent (James Petty)
• f0ea5b9 Add more benchmark for block position sizes (James Petty)
• 0280165 Record that the result of DictionaryBlock#copyPositions is compact (James Petty)
• 4871cee Opportunistically record computed dictionary size and uniqueIds (James Petty)
• e4f477c Add support for fixed width blocks computing their size more efficiently (James Petty)
• bb0ee5c Avoid redundant metastore calls for non-system Iceberg tables (Chunxu Tang)
• 7e138de Support distinct type in hive (Pranjal Shankhdhar)
• 40a79cd Make TestPinotSegmentPageSource single threaded (Neerad Somanchi)
• f63dfbc Add runtimeStats to track read calls to underlying storage (Nikhil Collooru)
• 9faf7d5 Support directly writing bytes to VariableWidthBlockBuilder (Arunachalam Thirupathi)
• 0233d34 Write data directly to OutputBuffer (Arunachalam Thirupathi)
• 807cb33 Avoid Slice allocation for Statistics (Arunachalam Thirupathi)
• 733d444 Generalize storage format in file metastore (Todd Gao)
• 2823595 Add some tests on serialization for file metastore (Todd Gao)
• 9a50892 Propagate session's connector properties in analyzeView (mengdilin)
• 85a5aa5 Add operators for distinct types (Pranjal Shankhdhar)
• 5c6efc7 Fix spelling (Josh Soref)
• 7c1b764 Fix out-of-date links in Cassandra connector doc (v-jizhang)
• f11dcb6 Fix incorrect type mapping in Cassandra documentation (v-jizhang)
• 1329da0 Fix typo in Cassandra doc (v-jizhang)
• 3020df1 implement secure_random function with uniform distribution (Darren Fu)
• 7b212bb Reduce hadoop configuration construction time (zhangyanbing)
• 0ebe1de Fix small inefficiency in StripeReader (Sergii Druzkin)
• 6e88468 Add OrcReaderOptions builder (Sergii Druzkin)
• 5eae5e5 Add adaptive execution policy (Tim Meehan)
• ba99b74 Add Thrift serde support for ServerInfo (Sergey Smirnov)
• 668d495 get raw filesystem should consider CachingFileSystem (guojianhua)
• 1788358 Add casts and operators for distinct types (Pranjal Shankhdhar)
• 8982bab Refactor lookupCast(). Use Type instead of TypeSignature (Pranjal Shankhdhar)
• ef1fd25 Improve listing performance of Hudi tables (Sagar Sumit)
• a9e6b6e Fix TestMemoryManager#testQueryMemoryPerNodeLimit flaky test (Swapnil Tailor)
• fce3301 Disable flaky TestDistributedQueryResource.testGetAllQueryInfoForLimits (Masha Basmanova)
• 143c6f6 Allow to pick a subset of TPC-DS tables to create in HiveQueryRunner (Masha Basmanova)
• c862457 Export DispatchManager counters (Mayank Garg)
• de18be1 Add TPCDS schema to HiveQueryRunner (George Wang)
• 172763f refactor logic to support create TPC tables for HiveQueryRunner (George Wang)
• 7830670 Throw proper error code when timezone is invalid from packDateTimeWithZone function. (Amit Dutta)
• 241adf1 Add join table reference check in JOIN ON clause (Ruslan Mardugalliamov)
• 761cd55 Fix cleanup workflow reference (Josh Soref)
• 8c50171 Align intermediate types for AVG aggregate function (Masha Basmanova)
• 4350f22 Handle the scalar subqueries for unnest (Maksim Dmitriyevich Podkorytov)
• e5c35a6 Fix cleanup CI (Josh Soref)
• e91cbb0 Add min compressible size to OrcOutputBuffer (Sergii Druzkin)
• 25b2f25 Ensure Parameters are in proper order for queries having WITH clause (v-jizhang)
• dd673a9 Fix bug in OrcOutputBuffer leading to severe undercompression of large blobs (Sergii Druzkin)
• ae9f559 Refactor iceberg multi-catalog smoke tests (Chunxu Tang)
• 2081ec9 Add getDefaultBlockSize and getDefaultReplication to CachingFileSystem (Chen)
• bcf13da Refactor null handling in ORC Dictionary writers (Arunachalam Thirupathi)
• f7774e5 Fixing TestServerInfoResource (Swapnil Tailor)
• 2ebf7ea Add distinct type definition (Pranjal Shankhdhar)
• e01d5ca Add ORC writer flush policy (Sergii Druzkin)
• b6eb0a0 Bumped Alluxio client to 2.7.3 (David N Perkins)
• 79370de Add a new JDBC driver parameter timeZoneId (v-jizhang)
• 3509718 Support optional row numbers in OrcSelectiveReader (Harsha Rastogi)
• aab1acf Add default implementation to HiveMetastore to fix the build (Beinan Wang)
• 7b83d6a Skip synthetic members to enable code coverage (Guy Moore)
• 39b6cea Remove the junit dependency to fix the build (Beinan Wang)
• b1402dc Create an abstract class for metadata of multiple iceberg catalogs (Chunxu Tang)
• e3c9af6 Enable cost based optimization by default (Rongrong Zhong)
• ca90676 Improve memory accounting for broadcast table (Arjun Gupta)
• 5edaef8 check field runtimeStats to prevent accessing undefined properties (ahouzheng)
• 2fb2c31 Add ability to do streaming for partial aggregation (Ke Wang)
• f66c85b Add ability to do streaming aggregation for hive table scans (Ke Wang)
• 39d20a8 Add ability to disable splitting file in hive connector (Ke Wang)
• afd4d2f fixing presto pinot doc for pinot catalog configs and add session property pinot.topn_large (Xiang Fu)
• 59d092c Support iceberg connector concurrent insertion (panyliu)
• fe1e84d Fixing Distributed Query Runner (Swapnil Tailor)
• 3cb1ee2 Skip reading Parquet pages using Column Indexes feature (Xinli shang)
• c911689 Add unit test for ThreadResource (Alan Xu)
• c07106b ThreadResource supports JSON serde at present. (Alan Xu)
• 6b5da9a Support json type in PinotSegmentPageSource (Xiang Fu)
• 4d8aad1 Fix parquet data page offset calculation in parquet writer (v-jizhang)

[NikhilCollooru]

Please follow the guidelines listed here for the wording ..
https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines#order-of-changes

[NikhilCollooru]

Please squash all the commits to one .

[tdcmeehan]

It would be good to figure out what this means. Perhaps "Fix error in Parquet writer".

[highker]

Do we have a user impact for this item? Like what query would fail before without the fix?

[asjadsyed]

Based on #17295: Trying to write a file larger than ~2 GB will results in an integer overflow exception

[asjadsyed]

Updated wording to

Fix integer overflow exception in Parquet writer when writing files larger than ~2 GB.

[rohanpednekar]

Folks, how long it will take to get this release published on prestodb.io? or is this blocked?