[SPARK-4295][External]Fix exception in SparkSinkSuite

[maji2014]

Handle exception in SparkSinkSuite, please refer to [SPARK-4295]

[AmplabJenkins]

Can one of the admins verify this patch?

[srowen]

Can you clarify how the tests pass if an exception is thrown? does that also need a fix?

[maji2014]

Exception throws before each test case although all test cases are passed . That's not the expect behavior i think. In order to avoid exception in each test case, set different channel name in each test case.
The behavior always appear when running SparkSinkSuite.scala directly
4/11/10 02:01:22 ERROR MonitoredCounterGroup: Failed to register monitored counter group for type: CHANNEL, name: null, javax.management.InstanceAlreadyExistsException: org.apache.flume.channel:type=null

[tdas]

@maji2014 Correct, I verified this error after adding a log4j properties in the external/flume-sink/src/test/resources and seeing the log4j logs. And manually verified that this test fixes the silent error.

As part of this patch, could you add the missing log4j.properties in external/flume-sink/src/test/resources, by copying it from streaming/src/test/resources ?

@harishreedharan you should take a look at this.

[tdas]

Why this crazy name? Since this is being only used in this class, this string will always be the same. In that case, we can simple do "Channel", isnt it?

[harishreedharan]

I think we can simply use a randomly generated string as the channel name, so the InstanceAlreadyExistsException would not pop up.

[maji2014]

As i have no good idea about how to get a random string name, such as how to get each test case name. In getClass.getDeclaredMethods.toString, we can get different string like[Ljava.lang.reflect.Method;@5f3ef269.

[tdas]

Haha, that's nifty, but hugely non-intuitive. Rather, please use scala.util.Random.nextString(10)

[tdas]

Also please address the other comment about adding log4j.properties.

[maji2014]

That's ok

[tdas]

Jenkins, this is ok to test

[SparkQA]

Test build #23212 has started for PR 3177 at commit 312620a.

• This patch merges cleanly.

[SparkQA]

Test build #23212 has finished for PR 3177 at commit 312620a.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23212/
Test PASSed.

[tdas]

@maji2014 Thanks for this fix. Merging this.