Generate Coral's RelNode for views from base table schema

[aastha25]

What changes are proposed in this pull request, and why are they necessary?

This PR includes the following fixes:
(1) Includes changes from #394 which aimed to remove CAST in HiveViewTable.toRel to solve Nested View Stale Schema Issue.
(2) Enables avro schema generation for a dataset with map field with NULL key
(3) When a field schema is a UnionType of a single data type, such as uniontype<array<string>>, surface the underlying field & its schema in Coral's SqlNode RelNode representation, such as array<string>. SingleUnionFieldReferenceTransformer has been introduced for backward compatibility.

For a table t1: (f1: uniontype ) and a view v1 defined on top of it SELECT custom_udf_to_remove_single_type_union(f1) AS f2 FROM t1
v1 has a schema: (f2: int)
The RelNode representation, prior to dropping the CAST operator in the RelNode, would force the derived data type to be int for f2, as expected. Hence, the generated avro schema could be analyzed by the Spark engine. However, after dropping the CAST, as documented here, the derived type is Struct<tag_0:int>. The Avro schema is incorrectly generated and cannot be analysed by the Spark engine.

The resolution is to derive the data type of single uniontype fields like f1 as the underlying data type (int in this case).

How was this patch tested?

./gradlew spotlessApply
./gradlew build
Updated UTs
Thoroughly tested production views across all supported engines.

[ljfgem]

@aastha25: Thanks for this PR. Could we create a PR for union type handling only and keep #394 ? Or keep the author information for the part in #394 if this PR can't be splitted?

[aastha25]

@ljfgem Unfortunately the PR cant be split up. All these changes need to go in together. I have updated the author's information for the commits related to #394.

[ljfgem]

Thank you @aastha25 for this PR!
Could you please add more details to the PR description and the code comment to clarify the necessity of the uniontype part changes? i.e. why it will fail if we don't make those changes, preferably with simple example?

[aastha25]

@ljfgem I have added an example clarifying the usecase in the PR description.

[ljfgem]

LGTM, thank you @aastha25 !

[wmoustafa]

Should not this PR change the behavior of CoalesceStructUtility so that if the input is a field with single type, it returns a struct? (e.g., extract_union(string) returns struct<tag_0: string>).

[aastha25]

This PR sets the return type of extract_union(string) as string, making the operator a no-op. I can modify the comment on the class to document this change if required.

[wmoustafa]

Discussed offline. We will create a follow-up PR to address this since the current approach: 1- breaks extract_union contract of returning only structs, 2- manages this new contract in another unrelated operator (DOT) with a method to guess the context.