Skip to content

[SPARK-20633][SQL] FileFormatWriter wrap the FetchFailedException which breaks job's failover#17893

Closed
lshmouse wants to merge 2 commits intoapache:masterfrom
lshmouse:FileFormatWriter
Closed

[SPARK-20633][SQL] FileFormatWriter wrap the FetchFailedException which breaks job's failover#17893
lshmouse wants to merge 2 commits intoapache:masterfrom
lshmouse:FileFormatWriter

Conversation

@lshmouse
Copy link
Copy Markdown

@lshmouse lshmouse commented May 8, 2017

What changes were proposed in this pull request?

Handle the fetch failed exception separately in FileFormatWriter.

How was this patch tested?

manual tests

Please review http://spark.apache.org/contributing.html before opening a pull request.

@lshmouse
Copy link
Copy Markdown
Author

lshmouse commented May 8, 2017

Pending. I am formatting the pull name~

@mridulm
Copy link
Copy Markdown
Contributor

mridulm commented May 8, 2017

This is not required anymore - post @squito's change.
Though it cant hurt I guess.

@lshmouse lshmouse changed the title FileFormatWriter wrap the FetchFailedException which breaks job's failover [SPARK-20633][SQL] FileFormatWriter wrap the FetchFailedException which breaks job's failover May 8, 2017
@squito
Copy link
Copy Markdown
Contributor

squito commented May 8, 2017

yeah, this isn't a bug anymore, but lets fix it -- cleaner error msg, and at least it will be easier for folks to see the fix in the future, as multiple people have stumbled on it and not realized it was fixed.

@lshmouse can you update the description to reference the fix in SPARK-19276?

@squito
Copy link
Copy Markdown
Contributor

squito commented May 8, 2017

Jenkins, ok to test

@SparkQA
Copy link
Copy Markdown

SparkQA commented May 8, 2017

Test build #76579 has finished for PR 17893 at commit c869d9c.

  • This patch fails SparkR unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@squito
Copy link
Copy Markdown
Contributor

squito commented May 8, 2017

Jenkins, retest this please

@SparkQA
Copy link
Copy Markdown

SparkQA commented May 8, 2017

Test build #76590 has finished for PR 17893 at commit c869d9c.

  • This patch fails SparkR unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@felixcheung
Copy link
Copy Markdown
Member

Please rebase to pick up fix for R tests

@squito
Copy link
Copy Markdown
Contributor

squito commented May 9, 2017

Jenkins, retest this please

@SparkQA
Copy link
Copy Markdown

SparkQA commented May 9, 2017

Test build #76689 has finished for PR 17893 at commit c869d9c.

  • This patch fails from timeout after a configured wait of `250m`.
  • This patch merges cleanly.
  • This patch adds no public classes.

@squito
Copy link
Copy Markdown
Contributor

squito commented May 30, 2017

since it seems @lshmouse is not updating this, I've resubmitted this here #18145, but all credit to the original author.

@asfgit asfgit closed this in d0f36bc May 31, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants