Improve call performance for sync_partition_metadata utility

[fgwang7w]

This PR fixes following issues:

Case 1: Current implementation for getting partition names takes partition values directly without taking into account of the actual path. e.g. The use of escaped character maybe encoded in HMS(Hive MetaStore) as partition name that could result incorrectness with current naming convention to fetch partition names.

• Solution: Retrieve partition storage location correctly from metastore using correct API calls

Case 2: Sync partition utility lists file status of each partition twice. In case of handling a large of number of partitions for an external table, redundant check file status is a performance bottleneck.

• Solution: Avoid file status check after knowing the path name from storage already by the sync partition procedure code path.

Case 3: For create partition call, batch size for number of trunks of partitions to be added to the metastore database is bounded by a fixed number. This limits the performance to update the metastore when the consumer of the underlying database is capable to handle a large number of batch and a large size of update batch.

• Solution: Different metastore systems could implement this commit batch size differently based on different underlying database capacity.

== RELEASE NOTES ==

General Changes
* improve performance for procedure sync_partition_metadata 

Hive Changes
* Increase default partition batch size per commit for Hive Metastore up to 10. 
* Set default partition batch size per commit for Glue Metastore up to 10,000

Detail troubleshoot and solution design can be found in this reference page here

# of partitions	Before (min)	after(min)	delta
2,526	5	1	5
12,000	54	6	9
39,493	92	12	7

[agrawalreetika]

Thank you for working on this.
The Benchmark number looks promising, this would be a good improvement for the sync-partition procedure.
I have taken the first pass on PR and added my comments.
These attached Benchmark results are from glue or HMS? Should we do it for both?

[imjalpreet]

@fgwang7w Thank you for the optimisation. I did the first pass and I have a question regarding one change added in SyncPartitionMetadataProcedure.

[imjalpreet]

I have one more suggestion, please let me know what are your views on the same.

[agrawalreetika]

The changes look good to me. Quick question, Are these attached Benchmark results from glue? Could we update the document with the same? Please update the PR release notes too about the partition batch.

[fgwang7w]

The changes look good to me. Quick question, Are these attached Benchmark results from glue? Could we update the document with the same? Please update the PR release notes too about the partition batch.

sure, will update the release notes. many thanks for help to review.

[ethanyzhang]

@tdcmeehan @prestodb/committers Hi Presto committers, this PR is reviewed internally already, can we have a second round review?

[yingsu00]

@fgwang7w George, great improvement! In addition to the comments in code, there are several other nits:

1. Commit title for 1 and 3 are too long
2. Commit title first letter shall be upper case
3. Can you use full sentences in the PR message?

[fgwang7w]

Thank you @yingsu00 for review. All comments are addressed, could you please give another pass? Thanks!
Thank you @pranjalssh @agrawalreetika for approving the PR.

[fgwang7w]

@prestodb/committers ping!

[yingsu00]

@fgwang7w Mostly good, just some nits.
In the PR message and commit messages:

without take into account

-> without taking into account

hms

-> HMS(Hive Meta Store)

Case 2: Sync partition utility lists file status of each partition twice. In case of handling a large of number of partitions for an external table, redundant check file status is a performance bottleneck.

Case 2: Sync partition utility lists file status of each partition twice, causing performance bottleneck when processing an external table with a large of number of partitions.

Commit 1 & 3 titles are too long.
Use upper case for the first letter in each sentence, including all commit titles, commit/PR messages, newly added comments.