Skip to content

Conversation

@pan3793
Copy link
Member

@pan3793 pan3793 commented Jan 12, 2024

What changes were proposed in this pull request?

Eliminate the HiveUtils.formatTimeVarsForHiveClient

Why are the changes needed?

HiveUtils.formatTimeVarsForHiveClient was introduced to handle compatibility with Hive prior 0.14.0, as Spark 4.0 only supports Hive 2.0+, it's now unnecessary.

Hive 0.14.0 introduces timeout operations in HiveConf, and changes default values of a bunch
of time ConfVars by adding time suffixes (s, ms, and d etc.). This breaks backwards-
compatibility when users are trying to connecting to a Hive metastore of lower version,
because these options are expected to be integral values in lower versions of Hive.

Here we enumerate all time ConfVars and convert their values to numeric strings according
to their output time units.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass GA.

Was this patch authored or co-authored using generative AI tooling?

No.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, this looks reasonable to me.

@dongjoon-hyun
Copy link
Member

Also, cc @LuciferYang

@HyukjinKwon
Copy link
Member

@pan3793
Copy link
Member Author

pan3793 commented Jan 13, 2024

@HyukjinKwon CI is green now

@dongjoon-hyun
Copy link
Member

Merged to master. Thank you, @pan3793 and all.

@LuciferYang
Copy link
Contributor

late LGTM ~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants