[SPARK-29902][Doc][Minor]Add listener event queue capacity configuration to documentation

[shahidki31]

What changes were proposed in this pull request?

Add listener event queue capacity configuration to documentation

Why are the changes needed?

We some time see many event drops happening in eventLog listener queue. So, instead of increasing all the queues size, using this config we just need to increase eventLog queue capacity.

scala> sc.parallelize(1 to 100000, 100000).count()
[Stage 0:=================================================>(98299 + 4) / 100000]19/11/14 20:56:35 ERROR AsyncEventQueue: Dropping event from queue eventLog. This likely means one of the listeners is too slow and cannot keep up with the rate at which tasks are being started by the scheduler.
19/11/14 20:56:35 WARN AsyncEventQueue: Dropped 1 events from eventLog since the application started.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing tests

[shahidki31]

cc @srowen Kindly review

[SparkQA]

Test build #113804 has finished for PR 26529 at commit 0a1fb5e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

"appStatus event queue hold events for internal application status listeners" is useful; "Capacity for appStatus event queue in Spark listener bus" seems sort of redundant. I'd maybe lead with "Capacity of the appStatus event queue, which holds events ..." just once. Here and elsewhere.

[shahidki31]

Thanks @srowen , I have updated

[srowen]

I think this is covered already by the fact that this is shown as the default here. You can remove it everywhere.

[shahidki31]

Done

[SparkQA]

Test build #113812 has finished for PR 26529 at commit acead31.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #113813 has finished for PR 26529 at commit 76180fa.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[shahidki31]

Retest this please

[SparkQA]

Test build #113816 has finished for PR 26529 at commit 76180fa.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[shahidki31]

retest this please

[SparkQA]

Test build #113853 has finished for PR 26529 at commit 76180fa.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[shahidki31]

I am not sure why the checks are failing. Seems unrelated error?

[srowen]

You can ignore the checks below at the moment. It is unrelated.

[srowen]

Merged to master

[shahidki31]

Thanks @srowen

[jiangxb1987]

#25307 (comment)
Is it safe to mention the queue names in the document?

[srowen]

Hm, good question. They're not explicitly marked internal, and I don't expect (?) they'll change. At the least maybe this could carry a comment that the names of the queues are subject to change.

[shahidki31]

@jiangxb1987 I think from the console/logs , user can get to know exactly from which queue event drop has happened and how many events dropped from that particular queue. Also user can see which event has dropped, if enable the trace.