Correct release notes entry in 377#12022
Conversation
|
|
||
| ## Hive connector | ||
|
|
||
| * Add support for partition names that include special characters. ({issue}`11719`) |
There was a problem hiding this comment.
This linked was linked from the one we now add and we thought this is fixed.
| * Add support for partition names that include special characters. ({issue}`11719`) | ||
| * Add support for `date` type partition names with timestamp formatting. ({issue}`11873`) | ||
| * Improve performance of queries that use Glue metadata. ({issue}`11869`) | ||
| * Fix `sync_partition_metadata` procedure when partition names differ from |
There was a problem hiding this comment.
This wording is still not ideal .. e.g. what was happening before .. did it just fail or corrupt something?
| * Add support for partition names that include special characters. ({issue}`11719`) | ||
| * Add support for `date` type partition names with timestamp formatting. ({issue}`11873`) | ||
| * Improve performance of queries that use Glue metadata. ({issue}`11869`) | ||
| * Fix `sync_partition_metadata` procedure when partition names differ from |
There was a problem hiding this comment.
sync_partition_metadata can cope with partition names which differ from the partition paths on the file system
There was a problem hiding this comment.
Thats not a full sentence .. also still missing what this did before
E.g.
Prevent failures of ...
There was a problem hiding this comment.
Fix the comparison mechanism for the partition names between the metastore and the file system in the system.sync_partition_metadata stored procedure.
Does this fit better?
There was a problem hiding this comment.
Well... still not sure what happened before.. did data get corrupted or did the procedure just fail or something else?
There was a problem hiding this comment.
Due to the fact that the comparison mechanism between the partition names stored in HMS and partition paths on the file system/ object storage was incorrect, when trying to sync the partition metadata the second time for a table, the user kept hitting rather puzzling exceptions.
When using 'FULL' mode after the first successful 'FULL' mode run:
dropping a partition added in the same transaction is not supported: testschema.test_table_external [2021-04-25, 20:00:13, 12345]
When using 'ADD' mode after the first successful 'FULL' mode run :
One or more partitions already exist for table 'testschema.test_table_external'
This was due to the fact that partition paths corresponding to existing partition names in HMS were falsely marked to be added/removed from HMS, reason why HMS threw the exceptions mentioned earlier.
I hope this comment gives a better context of the problem.
There was a problem hiding this comment.
Okay .. thats rather tricky. Do you have any suggestions on how to make a useful release notes entry out of this @martint ?
Maybe just
Prevent failures of the sync_partition_metadata system procedure for inconsistent partition name values stored in Hive metastore and used in the file system.
|
i thought there is an agreement already, sorry |
3 participants
Description
Correct a release notes entry for Trino 377 for the Hive connector where @martint and myself misread the comments and linked tickets.
Fix.
Docs
Fix a minor detail in the release notes.
Related issues, pull requests, and links
na.
Documentation
( ) No documentation is needed.
(x) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.
Release notes
(x) No release notes entries required.
( ) Release notes entries required with the following suggested text: