PARQUET-1990: [C++] Refuse to write ConvertedType::NA #9863
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
emkornfield
reviewed
Mar 31, 2021
cpp/src/parquet/thrift_internal.h
Outdated
Contributor
There was a problem hiding this comment.
DCHECK_Lt might be better here.
Member
Author
There was a problem hiding this comment.
I don't think it would change anything. Or we would have to do a full-blown check for valid values, which is a bit out of scope.
Contributor
There was a problem hiding this comment.
OK. I think it prevents accidental writes of UNDEFINED as well.
Member
Author
There was a problem hiding this comment.
Ah, I can add that as well :-)
Contributor
|
LGTM modulo one nit. |
pitrou
force-pushed
the
PARQUET-1990-invalid-converted-type
branch
from
777700e to
2efc70e
Compare
ConvertedType::NA corresponds to an invalid converted type that was once added to the Parquet spec: apache/parquet-format#45 but then quickly removed in favour of the Null logical type: apache/parquet-format#51 Unfortunately, Parquet C++ could still in some cases emit the unofficial converted type. Also remove the confusingly-named LogicalType::Unknown, while "UNKNOWN" in the Thrift specification points to LogicalType::Null.
pitrou
force-pushed
the
PARQUET-1990-invalid-converted-type
branch
from
2efc70e to
bfa5aec
Compare
Contributor
|
+1 thanks. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ConvertedType::NA corresponds to an invalid converted type that was once added to the Parquet spec:
apache/parquet-format#45
but then quickly removed in favour of the Null logical type:
apache/parquet-format#51
Unfortunately, Parquet C++ could still in some cases emit the unofficial converted type.
Also remove the confusingly-named LogicalType::Unknown, while "UNKNOWN" in the Thrift specification points to LogicalType::Null.