Skip to content

Conversation

@kiszk
Copy link
Member

@kiszk kiszk commented Jun 4, 2017

What changes were proposed in this pull request?

This PR waits for merging #18066

This PR adds compensation code for disabling the whole-stage codegen due to large number of columns.

When the whole-stage codegen is disabled, an iterator gets data from a row. Since the data is stored in columnar sotrage CachedBatch, the iterator is generated to copy data from the columnar storage to a row.
This PR supports only interger and double types. Another PR will support other data types.

How was this patch tested?

Added test suites into InMemoryColumnarQuerySuite

@SparkQA
Copy link

SparkQA commented Jun 4, 2017

Test build #77715 has finished for PR 18194 at commit 0c22730.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • class SpecificColumnarIterator extends $

@kiszk
Copy link
Member Author

kiszk commented Jun 4, 2017

Jenkins, retest this please

@SparkQA
Copy link

SparkQA commented Jun 4, 2017

Test build #77716 has finished for PR 18194 at commit 0c22730.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • class SpecificColumnarIterator extends $

@gatorsmile
Copy link
Member

gatorsmile commented Oct 28, 2017

@kiszk Could we close this PR? and you can re-submit it later.

@kiszk
Copy link
Member Author

kiszk commented Oct 28, 2017

Sure, it will not be used.

@kiszk kiszk closed this Oct 28, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants