Skip to content

Fix requiredField logic in StructSelectiveStreamReader#13884

Merged
mbasmanova merged 1 commit intoprestodb:masterfrom
bhhari:fix_struct_reader
Dec 21, 2019
Merged

Fix requiredField logic in StructSelectiveStreamReader#13884
mbasmanova merged 1 commit intoprestodb:masterfrom
bhhari:fix_struct_reader

Conversation

@bhhari
Copy link
Contributor

@bhhari bhhari commented Dec 20, 2019

This fixes the following exception

Caused by: java.lang.IllegalArgumentException: filter must be present if outputRequired is false
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:141)
	at com.facebook.presto.orc.reader.DoubleSelectiveStreamReader.<init>(DoubleSelectiveStreamReader.java:92)
	at com.facebook.presto.orc.reader.SelectiveStreamReaders.createStreamReader(SelectiveStreamReaders.java:70)
	at com.facebook.presto.orc.reader.StructSelectiveStreamReader.<init>(StructSelectiveStreamReader.java:164)
	at com.facebook.presto.orc.reader.SelectiveStreamReaders.createStreamReader(SelectiveStreamReaders.java:84)
	at com.facebook.presto.orc.OrcSelectiveRecordReader.createStreamReaders(OrcSelectiveRecordReader.java:581)
	at com.facebook.presto.orc.OrcSelectiveRecordReader.<init>(OrcSelectiveRecordReader.java:176)
	at com.facebook.presto.orc.OrcReader.createSelectiveRecordReader(OrcReader.java:234)
	at com.facebook.presto.hive.orc.OrcSelectivePageSourceFactory.createOrcPageSource(OrcSelectivePageSourceFactory.java:363)
== NO RELEASE NOTE ==

@bhhari bhhari requested a review from mbasmanova December 20, 2019 04:37
@bhhari bhhari added the aria Presto Aria performance improvements label Dec 20, 2019
@bhhari bhhari requested a review from highker December 20, 2019 04:37
Copy link
Contributor

@mbasmanova mbasmanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bhhari Thanks for fixing this.

@bhhari bhhari force-pushed the fix_struct_reader branch 2 times, most recently from 2cec773 to 5e03c0e Compare December 20, 2019 18:53
Read only the required sub-fields when the entire struct
column is not projected and there is a top level not null on it.
Current code is trying to read all nested fields and results in
an error "filter must be present if outputRequired is false"
Copy link

@highker highker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mbasmanova mbasmanova merged commit 5177bea into prestodb:master Dec 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aria Presto Aria performance improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants