Skip to content

Conversation

@cloud-fan
Copy link
Contributor

What changes were proposed in this pull request?

This is a followup of #30559 . The default parallelism config in Spark core is not good, as it's unclear where it applies. To not inherit this problem in Spark SQL, this PR refines the default parallelism SQL config, to make it clear that it only applies to leaf nodes.

Why are the changes needed?

Make the config clearer.

Does this PR introduce any user-facing change?

It changes an unreleased config.

How was this patch tested?

existing tests

@cloud-fan
Copy link
Contributor Author

@github-actions github-actions bot added the SQL label Dec 11, 2020
@SparkQA
Copy link

SparkQA commented Dec 11, 2020

Test build #132662 has finished for PR 30736 at commit 5473140.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you fix the build failure?

@SparkQA
Copy link

SparkQA commented Dec 15, 2020

Test build #132810 has finished for PR 30736 at commit f89e266.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 15, 2020

Test build #132816 has finished for PR 30736 at commit 2d82349.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 15, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37419/

@SparkQA
Copy link

SparkQA commented Dec 15, 2020

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37419/

@SparkQA
Copy link

SparkQA commented Dec 15, 2020

Test build #132817 has finished for PR 30736 at commit 7b05413.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan
Copy link
Contributor Author

cloud-fan commented Dec 15, 2020

thanks for the review, merging to master!

@cloud-fan cloud-fan closed this in 40c37d6 Dec 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants