Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[v1.9.x] [BUGFIX] Upgrade numpy to <1.20.0 to avoid security vulnerabilities affecting numpy<1.19.1 #20940

Merged
merged 8 commits into from
Mar 10, 2022

Conversation

DickJC123
Copy link
Contributor

@DickJC123 DickJC123 commented Mar 7, 2022

Description

There are vulnerabilities reported in numpy versions currently used in our CI system, as mentioned in issue #20869:

https://nvd.nist.gov/vuln/detail/CVE-2021-41495
https://nvd.nist.gov/vuln/detail/CVE-2021-41496

These issues require a numpy version >= 1.19.1 to avoid. This PR takes the somewhat conservative approach of only advancing the numpy version to 1.19.5 (actually <1.20.0) in order to avoid the issues, without advancing numpy farther and risking creating additional issues. Note that the numpy version is not restricted to >=1.19.1, since some of the build and test environments do not support numpy 1.19. Thus, with this PR, MXNet will be built to avoid the vulnerabilities if the system supports it.

The unittests that started failing on numpy 1.19 were test_np_delete and test_np_array_function_protocol, due to the change in the way numpy 1.19 delete() interprets the indices-to-delete when supplied as an array. The tests were modified to pass for all versions of numpy, both 1.19 and earlier. See https://numpy.org/doc/stable/release.html for more details.

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

@mxnet-bot
Copy link

Hey @DickJC123 , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [edge, website, windows-cpu, clang, centos-cpu, unix-cpu, sanity, centos-gpu, windows-gpu, unix-gpu, miscellaneous]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@mseth10 mseth10 added the pr-work-in-progress PR is still work in progress label Mar 7, 2022
@DickJC123 DickJC123 changed the title [v1.9.x] [WIP] [BUGFIX] Upgrade numpy>=1.19.1 to avoid security vulnerabilities [v1.9.x] [BUGFIX] Upgrade numpy to <1.20.0 to avoid security vulnerabilities affecting numpy<1.19.1 Mar 9, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Mar 9, 2022
@DickJC123
Copy link
Contributor Author

FYI, issue #20869 discussing the numpy upgrade motivation had the comment from @huubvh95:

Is it also possible to update Graphviz in the PR? The required version for Graphviz is rather old.
See issue https://github.com/apache/incubator-mxnet/issues/20897

My feeling on this:

I'm not able to do a reasonable job of testing an updated version of graphviz.
If we put the graphviz update in this PR, then a required revert based on either a
numpy or graphviz problem would revert both modules- an unfortunate coupling.

Could you suggest a graphviz version to move to, based on some testing, and make a separate PR for that?

Copy link
Contributor

@josephevans josephevans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@DickJC123 DickJC123 requested a review from szha March 9, 2022 20:22
Copy link
Contributor

@samskalicky samskalicky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks @DickJC123 !

@DickJC123 DickJC123 merged commit 67467f8 into apache:v1.9.x Mar 10, 2022
DickJC123 added a commit to DickJC123/mxnet that referenced this pull request Mar 12, 2022
…ilities affecting numpy<1.19.1 (apache#20940)

* Pin numpy==1.19.1 to demonstrate issues

* Relax min numpy version

* Make test_np_array_function_protocol delete() testing work with numpy>=1.19

* Fix test_np_delete to also work with numpy>=1.19

* Pip install python module 'packaging'

* More pip-install 'packaging'

* Update windows requirements.txt

* Allow numpy as advanced as 1.19.5
DickJC123 added a commit to DickJC123/mxnet that referenced this pull request Mar 14, 2022
…ilities affecting numpy<1.19.1 (apache#20940)

* Pin numpy==1.19.1 to demonstrate issues

* Relax min numpy version

* Make test_np_array_function_protocol delete() testing work with numpy>=1.19

* Fix test_np_delete to also work with numpy>=1.19

* Pip install python module 'packaging'

* More pip-install 'packaging'

* Update windows requirements.txt

* Allow numpy as advanced as 1.19.5
DickJC123 added a commit that referenced this pull request Mar 18, 2022
* [v1.9.x] [BUGFIX] Upgrade numpy to <1.20.0 to avoid security vulnerabilities affecting numpy<1.19.1 (#20940)

* Pin numpy==1.19.1 to demonstrate issues

* Relax min numpy version

* Make test_np_array_function_protocol delete() testing work with numpy>=1.19

* Fix test_np_delete to also work with numpy>=1.19

* Pip install python module 'packaging'

* More pip-install 'packaging'

* Update windows requirements.txt

* Allow numpy as advanced as 1.19.5

* Trigger CI
DickJC123 added a commit that referenced this pull request Mar 18, 2022
#20957)

* Stop skipping tests mentioned in issue 18600

* Reenable test_np_random_chisquare also

* [v1.9.x] [BUGFIX] Upgrade numpy to <1.20.0 to avoid security vulnerabilities affecting numpy<1.19.1 (#20940)

* Pin numpy==1.19.1 to demonstrate issues

* Relax min numpy version

* Make test_np_array_function_protocol delete() testing work with numpy>=1.19

* Fix test_np_delete to also work with numpy>=1.19

* Pip install python module 'packaging'

* More pip-install 'packaging'

* Update windows requirements.txt

* Allow numpy as advanced as 1.19.5

* Fix test_np_random_{beta,f,chisquare}
@waytrue17 waytrue17 mentioned this pull request Apr 12, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
pr-awaiting-review PR is waiting for code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants