PARQUET-2: Adding Type Persuasion for Primitive Types

[danielcweeks]

Original from the old repo: https://github.com/Parquet/parquet-mr/pull/410
JIRA: https://issues.apache.org/jira/browse/PARQUET-2

These changes allow primitive types to be requested as different types than what is stored in the file format using a flag to turn off strict type checking (default is on). Types are cast to the requested type where possible and will suffer precision loss for casting where necessary (e.g. requesting a double as an int).

No performance penalty is imposed for using the type defined in the file type. A flag exists to

A 6x6 test case is provided to test conversion between the primitive types.

[julienledem]

We should put the default implementation here:

protected Type union(Type toMerge) {
    return union(toMerge, true);
}

[danielcweeks]

I tried to put the default implementation in the abstract class, but the
maven enforcer plugin wouldn't allow me to do it. I assume removing the
abstract is considered an interface change.

On Fri, Jun 20, 2014 at 9:21 PM, Julien Le Dem [email protected]
wrote:

In parquet-column/src/main/java/parquet/schema/Type.java:

@@ -195,6 +195,13 @@ public boolean equals(Object other) {
* @return the union result of merging toMerge into this
*/
protected abstract Type union(Type toMerge);

We should put the default implementation here:

protected Type union(Type toMerge) {
return union(toMerge, true);
}

—
Reply to this email directly or view it on GitHub
https://github.com/apache/incubator-parquet-mr/pull/3/files#r14048282.

[julienledem]

Thanks Daniel!
I made a few comments.
Otherwise this LGTM!

[julienledem]

please prefix the PR title with "PARQUET-2: "

[julienledem]

Thank you @dcw-netflix !