[SPARK-28763][SQL][TEST] Flaky Tests: SparkThriftServerProtocolVersionsSuite.HIVE_CLI_SERVICE_PROTOCOL_V1 get binary type #25480

wangyum · 2019-08-16T23:56:47Z

What changes were proposed in this pull request?

This pr try to fix flaky tests:

org.scalatest.exceptions.TestFailedException: "[?](" did not equal "[�]("

org.apache.spark.sql.hive.thriftserver.SparkThriftServerProtocolVersionsSuite.HIVE_CLI_SERVICE_PROTOCOL_V1 get binary type | 0.48 秒 | 2
-- | -- | --
org.apache.spark.sql.hive.thriftserver.SparkThriftServerProtocolVersionsSuite.HIVE_CLI_SERVICE_PROTOCOL_V2 get binary type | 0.34 秒 | 2
org.apache.spark.sql.hive.thriftserver.SparkThriftServerProtocolVersionsSuite.HIVE_CLI_SERVICE_PROTOCOL_V3 get binary type | 0.36 秒 | 2
org.apache.spark.sql.hive.thriftserver.SparkThriftServerProtocolVersionsSuite.HIVE_CLI_SERVICE_PROTOCOL_V4 get binary type | 0.3 秒 | 2
org.apache.spark.sql.hive.thriftserver.SparkThriftServerProtocolVersionsSuite.HIVE_CLI_SERVICE_PROTOCOL_V5 get binary type | 0.36 秒 | 2

https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4832/testReport/

How was this patch tested?

N/A

wangyum · 2019-08-17T00:04:46Z

retest this please

SparkQA · 2019-08-17T00:25:51Z

Test build #109253 has finished for PR 25480 at commit 9dc7163.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-08-17T00:29:36Z

Test build #109254 has finished for PR 25480 at commit 9dc7163.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

wangyum · 2019-08-17T01:40:14Z

retest this please

wangyum · 2019-08-17T01:40:18Z

retest this please

HyukjinKwon · 2019-08-17T02:02:11Z

...st/scala/org/apache/spark/sql/hive/thriftserver/SparkThriftServerProtocolVersionsSuite.scala

      }
-      testExecuteStatementWithProtocolVersion(version, "SELECT cast(49960 as binary)") { rs =>
+      testExecuteStatementWithProtocolVersion(version,
+        "SELECT cast(cast(49960 as int) as binary)") { rs =>


@wangyum Btw how does this fix the flakiness?

SparkQA · 2019-08-17T02:06:15Z

Test build #109256 has finished for PR 25480 at commit 9dc7163.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2019-08-17T02:11:20Z

Thank you for the fix. I also have the same question like @HyukjinKwon .
Could you describe more about the root cause in the PR description?

wangyum · 2019-08-17T04:12:04Z

...st/scala/org/apache/spark/sql/hive/thriftserver/SparkThriftServerProtocolVersionsSuite.scala

-        assert(rs.getString(1) === UTF8String.fromBytes(NumberConverter.toBinary(49960)).toString)
+        assertResult(Array(0, 0, 65533, 40).map(_.toChar)) {
+          rs.getString(1).toCharArray
+        }


@dongjoon-hyun @HyukjinKwon Maybe we should uses CharArray.

What is the difference? Could you tell us the root cause you think?

I'm not sure. May be related to environment:
https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4832/
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109194/testReport/

These two PullRequestBuilders are built on the same machine(amp-jenkins-worker-03). The first one failed, but the last one successful. The environment differences:

I will verify it later.

Maybe, something like LANG=en_US.UTF_8?

@shaneknapp I'd set it everywhere, personally, if it's easy.

it's easy, but requires each worker to be disconnected/reconnected... i'll get that set up now but it won't go in to effect until i have the opportunity to restart things.

Thank you @shaneknapp

Could we close this PR?

Thank you for investigating this so far, @wangyum . Yes. It looks like that.
You can close this and the JIRA together. If the failure is reported again, we can reopen this.

SparkQA · 2019-08-17T04:29:41Z

Test build #109261 has finished for PR 25480 at commit da84f92.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2019-08-17T08:20:10Z

@wangyum . In the same test log, there are 8 protocols and V6/V7/V8 passed with assert(rs.getString(1) === UTF8String.fromBytes(NumberConverter.toBinary(49960)).toString). For all 8 protocols, the assertion was the same and shared the same machine environment for V1~V8. So, the assertion looks correct to me.

[info] - HIVE_CLI_SERVICE_PROTOCOL_V1 get binary type *** FAILED *** (486 milliseconds)
[info] - HIVE_CLI_SERVICE_PROTOCOL_V2 get binary type *** FAILED *** (345 milliseconds)
[info] - HIVE_CLI_SERVICE_PROTOCOL_V3 get binary type *** FAILED *** (361 milliseconds)
[info] - HIVE_CLI_SERVICE_PROTOCOL_V4 get binary type *** FAILED *** (304 milliseconds)
[info] - HIVE_CLI_SERVICE_PROTOCOL_V5 get binary type *** FAILED *** (360 milliseconds)
[info] - HIVE_CLI_SERVICE_PROTOCOL_V6 get binary type (521 milliseconds)
[info] - HIVE_CLI_SERVICE_PROTOCOL_V7 get binary type (588 milliseconds)
[info] - HIVE_CLI_SERVICE_PROTOCOL_V8 get binary type (565 milliseconds)

## What changes were proposed in this pull request? This PR fix Hive 0.12 JDBC client can not handle binary type: ```sql Connected to: Hive (version 3.0.0-SNAPSHOT) Driver: Hive (version 0.12.0) Transaction isolation: TRANSACTION_REPEATABLE_READ Beeline version 0.12.0 by Apache Hive 0: jdbc:hive2://localhost:10000> SELECT cast('ABC' as binary); Error: java.lang.ClassCastException: [B incompatible with java.lang.String (state=,code=0) ``` Server log: ``` 19/08/07 10:10:04 WARN ThriftCLIService: Error fetching results: java.lang.RuntimeException: java.lang.ClassCastException: [B incompatible with java.lang.String at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:83) at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) at java.security.AccessController.doPrivileged(AccessController.java:770) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) at com.sun.proxy.$Proxy26.fetchResults(Unknown Source) at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:455) at org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:621) at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553) at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:53) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:310) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:819) Caused by: java.lang.ClassCastException: [B incompatible with java.lang.String at org.apache.hive.service.cli.ColumnValue.toTColumnValue(ColumnValue.java:198) at org.apache.hive.service.cli.RowBasedSet.addRow(RowBasedSet.java:60) at org.apache.hive.service.cli.RowBasedSet.addRow(RowBasedSet.java:32) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.$anonfun$getNextRowSet$1(SparkExecuteStatementOperation.scala:151) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$Lambda$1923.000000009113BFE0.apply(Unknown Source) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withSchedulerPool(SparkExecuteStatementOperation.scala:299) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.getNextRowSet(SparkExecuteStatementOperation.scala:113) at org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:220) at org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:785) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) ... 18 more ``` ## How was this patch tested? unit tests Closes #25379 from wangyum/SPARK-28474. Authored-by: Yuming Wang <[email protected]> Signed-off-by: HyukjinKwon <[email protected]>

try 1

9dc7163

HyukjinKwon reviewed Aug 17, 2019

View reviewed changes

dongjoon-hyun added SQL TESTS labels Aug 17, 2019

try 2

da84f92

wangyum commented Aug 17, 2019

View reviewed changes

HyukjinKwon mentioned this pull request Aug 17, 2019

[SPARK-28474][SQL] Hive 0.12 JDBC client can not handle binary type #25379

Closed

wangyum closed this Aug 22, 2019

HyukjinKwon mentioned this pull request Mar 25, 2020

[SPARK-30301][SQL] Fix wrong results when datetimes as fields of complex types #26942

Closed

[SPARK-28763][SQL][TEST] Flaky Tests: SparkThriftServerProtocolVersionsSuite.HIVE_CLI_SERVICE_PROTOCOL_V1 get binary type #25480

[SPARK-28763][SQL][TEST] Flaky Tests: SparkThriftServerProtocolVersionsSuite.HIVE_CLI_SERVICE_PROTOCOL_V1 get binary type #25480

Uh oh!

Conversation

wangyum commented Aug 16, 2019 • edited by dongjoon-hyun Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

wangyum commented Aug 17, 2019

Uh oh!

SparkQA commented Aug 17, 2019

Uh oh!

SparkQA commented Aug 17, 2019

Uh oh!

wangyum commented Aug 17, 2019

Uh oh!

wangyum commented Aug 17, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 17, 2019

Uh oh!

dongjoon-hyun commented Aug 17, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 17, 2019

Uh oh!

dongjoon-hyun commented Aug 17, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

wangyum commented Aug 16, 2019 •

edited by dongjoon-hyun

Loading

dongjoon-hyun commented Aug 17, 2019 •

edited

Loading