Skip to content

Conversation

@zhouyifan279
Copy link
Contributor

@zhouyifan279 zhouyifan279 commented Dec 8, 2022

What changes were proposed in this pull request?

Remove hive-vector-code-gen and its dependent jars from spark distribution

Why are the changes needed?

hive-vector-code-gen is not used in spark

Remove it to avoid vulnerability scanners's alert.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Passed current test cases

@github-actions github-actions bot added the BUILD label Dec 8, 2022
@LuciferYang
Copy link
Contributor

cc @sunchao FYI

@LuciferYang
Copy link
Contributor

@zhouyifan279 Title should be [SPARK-39948][BUILD] Exclude...

@github-actions github-actions bot added the SQL label Dec 8, 2022
@zhouyifan279 zhouyifan279 changed the title [SPARK-39948] Exclude hive-vector-code-gen dependency to solve CVE-2020-13936 [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency to solve CVE-2020-13936 Dec 8, 2022
@zhouyifan279
Copy link
Contributor Author

@zhouyifan279 Title should be [SPARK-39948][BUILD] Exclude...

Thanks, corrected.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. (Pending CIs)

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, @zhouyifan279 . The GitHub Action is not triggerred at your repository still.
Screenshot 2022-12-08 at 12 58 25 AM

@zhouyifan279
Copy link
Contributor Author

BTW, @zhouyifan279 . The GitHub Action is not triggerred at your repository still. Screenshot 2022-12-08 at 12 58 25 AM

@dongjoon-hyun thanks for your reminding. Several workflows were triggered by previous commits. I have cancelled them.

@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@srowen
Copy link
Member

srowen commented Dec 8, 2022

If we can exclude this code, then, the CVE doesn't affect Spark to begin with right?

Copy link
Member

@sunchao sunchao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

"org.apache.curator:*",
"org.pentaho:pentaho-aggdesigner-algorithm"))
"org.pentaho:pentaho-aggdesigner-algorithm",
"org.apache.hive:hive-vector-code-gen"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do other Hive versions like v2_2 have the same issue?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhouyifan279 Could you answer @sunchao 's question?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sunchao According to this commit in Hive Project, hive-vector-code-gen is added as hive-exec dependency since hive-2.3.0.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for confirming, @zhouyifan279 .

@dongjoon-hyun
Copy link
Member

Yes, right, @srowen . Apache Spark is not affected.

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency to solve CVE-2020-13936 [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency Dec 8, 2022
@dongjoon-hyun
Copy link
Member

I revised the PR title and description by removing CVE info, @srowen .

@dongjoon-hyun
Copy link
Member

Merged to master branch for Apache Spark 3.4.0.

@dongjoon-hyun
Copy link
Member

What is your JIRA ID, @zhouyifan279 ?

@dongjoon-hyun
Copy link
Member

I found your JIRA ID, zhouyifan279, and added it to Apache Spark contributor group.

Also, from the commit log, I found the following three JIRAs and assigned them to you.

Thank you for your contribution, @zhouyifan279 .

@zhouyifan279
Copy link
Contributor Author

What is your JIRA ID, @zhouyifan279 ?

I reused this issue: https://issues.apache.org/jira/browse/SPARK-39948

@zhouyifan279
Copy link
Contributor Author

I found your JIRA ID, zhouyifan279, and added it to Apache Spark contributor group.

Also, from the commit log, I found the following three JIRAs and assigned them to you.

Thank you for your contribution, @zhouyifan279 .

Thanks. It's my pleasure to do something for the Spark community.

beliefer pushed a commit to beliefer/spark that referenced this pull request Dec 18, 2022
### What changes were proposed in this pull request?
Remove hive-vector-code-gen and its dependent jars from spark distribution

### Why are the changes needed?
hive-vector-code-gen is not used in spark

Remove it to avoid vulnerability scanners's alert.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Passed current test cases

Closes apache#38978 from zhouyifan279/SPARK-39948.

Authored-by: zhouyifan279 <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
@zhouyifan279 zhouyifan279 deleted the SPARK-39948 branch August 14, 2023 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants