SPARK-2701: ConnectionManager throws out of "Could not find reference for received ack message xxx" exception.

[witgo]

No description provided.

[SparkQA]

QA tests have started for PR 1603. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17221/consoleFull

[SparkQA]

QA results for PR 1603:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17221/consoleFull

[SparkQA]

QA tests have started for PR 1603. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17223/consoleFull

[aarondav]

Does the movement of the status modification into the synchronized block change anything? It seems the only effective change here is downgrading an exception to a log message.

Because we remove the status from the map within the synchronized block, it would seem that it's safe to do our subsequent modification of the status without (though it does seem to be more consistent with the rest of the code to do it within).

[witgo]

Throw an exception here cause System.exit(ExecutorExitCode.UNCAUGHT_EXCEPTION) is called.
This is not necessary.

[SparkQA]

QA results for PR 1603:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17223/consoleFull

[mridulm]

I have not analyzed in detail, but this looks like something which might cause deadlocks ..
Changing to logging instead of throwing exception is a good idea though : earlier we were not exit'ing in case of uncaught exception - which was introduced in some subsequent release - so this should be done now (logError).

[JoshRosen]

How did you trigger this exception? I wonder whether this points to a synchronization error somewhere, or a bug in how the ACK'er handles message ids.

[witgo]

This appeared in a production environment. I don't know how to reproduce it.

[SparkQA]

QA tests have started for PR 1603. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18092/consoleFull

[SparkQA]

QA results for PR 1603:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18092/consoleFull

[SparkQA]

QA tests have started for PR 1603. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18093/consoleFull

[SparkQA]

QA results for PR 1603:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18093/consoleFull

[SparkQA]

QA tests have started for PR 1603. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18098/consoleFull

[SparkQA]

QA results for PR 1603:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18098/consoleFull