Skip to content

Conversation

@dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Jun 28, 2016

What changes were proposed in this pull request?

After SPARK-15674, DDLStrategy prints out the following deprecation messages in the testsuites.

12:10:53.284 WARN org.apache.spark.sql.execution.SparkStrategies$DDLStrategy:
CREATE TEMPORARY TABLE normal_orc_source USING... is deprecated,
please use CREATE TEMPORARY VIEW viewName USING... instead

Total : 40

  • JDBCWriteSuite: 14
  • DDLSuite: 6
  • TableScanSuite: 6
  • ParquetSourceSuite: 5
  • OrcSourceSuite: 2
  • SQLQuerySuite: 2
  • HiveCommandSuite: 2
  • JsonSuite: 1
  • PrunedScanSuite: 1
  • FilteredScanSuite 1

This PR replaces CREATE TEMPORARY TABLE with CREATE TEMPORARY VIEW in order to remove the deprecation messages in the above testsuites except DDLSuite, SQLQuerySuite, HiveCommandSuite.

The Jenkins results shows only remaining 10 messages.

https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61422/consoleFull

How was this patch tested?

This is a testsuite-only change.

sql(
s"""
|CREATE TEMPORARY TABLE PEOPLE
|CREATE OR REPLACE TEMPORARY VIEW PEOPLE
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This testsuite needs CREATE OR REPLACE.

@SparkQA
Copy link

SparkQA commented Jun 29, 2016

Test build #61422 has finished for PR 13956 at commit 1eef876.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-16267][TEST] Replace deprecated CREATE TEMPORARY TABLE from testsuites. [SPARK-16267][TEST] Replace deprecated CREATE TEMPORARY TABLE ... USING from testsuites. Jun 29, 2016
@dongjoon-hyun
Copy link
Member Author

Could you review this PR, @srowen ?

@dongjoon-hyun
Copy link
Member Author

dongjoon-hyun commented Jun 29, 2016

Hi, @rxin .

Today, I faced the following irrelevant error in my other PR (inline PR), #13976.

Exception encountered when attempting to run a suite with class name: org.apache.spark.sql.jdbc.JDBCWriteSuite *** ABORTED *** (886 milliseconds)
[info]   org.h2.jdbc.JdbcSQLException: Schema "TEST" already exists; SQL statement:

https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61479/consoleFull

I think this PR can help the flaky testsuite since this PR replace it with CREATE OR REPLACE TEMPORARY VIEW PEOPLE.

@rxin
Copy link
Contributor

rxin commented Jun 29, 2016

Can you roll back the license whitespace changes?

@dongjoon-hyun
Copy link
Member Author

Yep. I reverted the License indentation fixes.

@rxin
Copy link
Contributor

rxin commented Jun 29, 2016

LGTM pending Jenkins.

@SparkQA
Copy link

SparkQA commented Jun 30, 2016

Test build #61502 has finished for PR 13956 at commit 6c69161.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin
Copy link
Contributor

rxin commented Jun 30, 2016

Merging in master/2.0.

asfgit pushed a commit that referenced this pull request Jun 30, 2016
…ING` from testsuites.

## What changes were proposed in this pull request?

After SPARK-15674, `DDLStrategy` prints out the following deprecation messages in the testsuites.

```
12:10:53.284 WARN org.apache.spark.sql.execution.SparkStrategies$DDLStrategy:
CREATE TEMPORARY TABLE normal_orc_source USING... is deprecated,
please use CREATE TEMPORARY VIEW viewName USING... instead
```

Total : 40
- JDBCWriteSuite: 14
- DDLSuite: 6
- TableScanSuite: 6
- ParquetSourceSuite: 5
- OrcSourceSuite: 2
- SQLQuerySuite: 2
- HiveCommandSuite: 2
- JsonSuite: 1
- PrunedScanSuite: 1
- FilteredScanSuite  1

This PR replaces `CREATE TEMPORARY TABLE` with `CREATE TEMPORARY VIEW` in order to remove the deprecation messages in the above testsuites except `DDLSuite`, `SQLQuerySuite`, `HiveCommandSuite`.

The Jenkins results shows only remaining 10 messages.

https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61422/consoleFull

## How was this patch tested?

This is a testsuite-only change.

Author: Dongjoon Hyun <[email protected]>

Closes #13956 from dongjoon-hyun/SPARK-16267.

(cherry picked from commit 831a04f)
Signed-off-by: Reynold Xin <[email protected]>
@asfgit asfgit closed this in 831a04f Jun 30, 2016
@dongjoon-hyun
Copy link
Member Author

Thank you!

@dongjoon-hyun dongjoon-hyun deleted the SPARK-16267 branch July 20, 2016 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants