Skip to content

Conversation

@maropu
Copy link
Member

@maropu maropu commented Mar 9, 2021

What changes were proposed in this pull request?

This PR intends to remove unnecessary SQLConf.withExistingConf in CastSuite; since we've remove ParVector in #31775, we no longer need to copy SQL configs into each thread env.

Why are the changes needed?

Clean up the code.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Run the existing tests.

@github-actions github-actions bot added the SQL label Mar 9, 2021
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for making a follow-up, @maropu .

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-33498][SQL][FOLLOWUP] Remove SQLConf.withExistingConf in CastSuite [SPARK-33498][SQL][TESTS][FOLLOWUP] Remove SQLConf.withExistingConf in CastSuite Mar 9, 2021
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM.
This is a test-only PR and I verified manually.

[info] All tests passed.
[info] Passed: Total 4879, Failed 0, Errors 0, Passed 4879, Ignored 5
[success] Total time: 369 s (06:09), completed Mar 8, 2021 11:51:30 PM
build/sbt "catalyst/test"  1393.45s user 42.65s system 380% cpu 6:16.93 total

Merged to master.

@dongjoon-hyun
Copy link
Member

There is a conflict on branch-3.1. Could you make a backporting PR?

@dongjoon-hyun
Copy link
Member

Oh, BTW, @maropu .
I realized that this is a follow-up of old SPARK-33498 who fixed version is 3.1.0. It looks a little weird because this goes only to 3.2.0 and 3.1.2.

@SparkQA
Copy link

SparkQA commented Mar 9, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40485/

@SparkQA
Copy link

SparkQA commented Mar 9, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40485/

@maropu
Copy link
Member Author

maropu commented Mar 9, 2021

There is a conflict on branch-3.1. Could you make a backporting PR?

Thanks for your quick review&merge! I'll open a PR for that later.

I realized that this is a follow-up of old SPARK-33498 who fixed version is 3.1.0. It looks a little weird because this goes only to 3.2.0 and 3.1.2.

Ah, yea, as you said, it looks a bit weird. But, this is a minor update, so IMO it is okay as it is. Or, do you think we need to do anything for that?

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Mar 9, 2021

It's not a problem. I thought it was a follow-up of SPARK-34660 . But, according to this PR's content, it's more proper as a follow-up of SPARK-33498 . Maybe, next time, we had better create a new JIRA for a released patch because the Fixed Version are not the same. For this one, there is nothing for us to do more~

dongjoon-hyun pushed a commit that referenced this pull request Mar 12, 2021
…onf in CastSuite

### What changes were proposed in this pull request?

This PR intends to remove unnecessary `SQLConf.withExistingConf` in `CastSuite`; since we've remove `ParVector ` in #31775, we no longer need to copy SQL configs into each thread env.

This PR is a backport of #31785.

### Why are the changes needed?

Clean up the code.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Run the existing tests.

Closes #31813 from maropu/SPARK-33498-BRANCH3.1.

Authored-by: Takeshi Yamamuro <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
flyrain pushed a commit to flyrain/spark that referenced this pull request Sep 21, 2021
…onf in CastSuite

### What changes were proposed in this pull request?

This PR intends to remove unnecessary `SQLConf.withExistingConf` in `CastSuite`; since we've remove `ParVector ` in apache#31775, we no longer need to copy SQL configs into each thread env.

This PR is a backport of apache#31785.

### Why are the changes needed?

Clean up the code.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Run the existing tests.

Closes apache#31813 from maropu/SPARK-33498-BRANCH3.1.

Authored-by: Takeshi Yamamuro <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants