Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

[v1.9.x] [BUGFIX] Upgrade numpy to <1.20.0 to avoid security vulnerabilities affecting numpy<1.19.1#20940

Merged
DickJC123 merged 8 commits intoapache:v1.9.xfrom
DickJC123:upgrade_numpy_to_1.19.1
Mar 10, 2022
Merged

[v1.9.x] [BUGFIX] Upgrade numpy to <1.20.0 to avoid security vulnerabilities affecting numpy<1.19.1#20940
DickJC123 merged 8 commits intoapache:v1.9.xfrom
DickJC123:upgrade_numpy_to_1.19.1

Conversation

@DickJC123
Copy link
Contributor

@DickJC123 DickJC123 commented Mar 7, 2022 •
edited
Loading

Description

There are vulnerabilities reported in numpy versions currently used in our CI system, as mentioned in issue #20869:

https://nvd.nist.gov/vuln/detail/CVE-2021-41495
https://nvd.nist.gov/vuln/detail/CVE-2021-41496

These issues require a numpy version >= 1.19.1 to avoid. This PR takes the somewhat conservative approach of only advancing the numpy version to 1.19.5 (actually <1.20.0) in order to avoid the issues, without advancing numpy farther and risking creating additional issues. Note that the numpy version is not restricted to >=1.19.1, since some of the build and test environments do not support numpy 1.19. Thus, with this PR, MXNet will be built to avoid the vulnerabilities if the system supports it.

The unittests that started failing on numpy 1.19 were test_np_delete and test_np_array_function_protocol, due to the change in the way numpy 1.19 delete() interprets the indices-to-delete when supplied as an array. The tests were modified to pass for all versions of numpy, both 1.19 and earlier. See https://numpy.org/doc/stable/release.html for more details.

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

@mxnet-bot
Copy link

mxnet-bot commented Mar 7, 2022

Hey @DickJC123 , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [edge, website, windows-cpu, clang, centos-cpu, unix-cpu, sanity, centos-gpu, windows-gpu, unix-gpu, miscellaneous]

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@mseth10 mseth10 added the pr-work-in-progress PR is still work in progress label Mar 7, 2022
@DickJC123 DickJC123 changed the title [v1.9.x] [WIP] [BUGFIX] Upgrade numpy>=1.19.1 to avoid security vulnerabilities [v1.9.x] [BUGFIX] Upgrade numpy to <1.20.0 to avoid security vulnerabilities affecting numpy<1.19.1 Mar 9, 2022
@DickJC123 DickJC123 requested a review from josephevans March 9, 2022 02:07
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Mar 9, 2022
@DickJC123
Copy link
Contributor Author

DickJC123 commented Mar 9, 2022

FYI, issue #20869 discussing the numpy upgrade motivation had the comment from @huubvh95:

Is it also possible to update Graphviz in the PR? The required version for Graphviz is rather old.
See issue https://github.com/apache/incubator-mxnet/issues/20897

My feeling on this:

I'm not able to do a reasonable job of testing an updated version of graphviz.
If we put the graphviz update in this PR, then a required revert based on either a
numpy or graphviz problem would revert both modules- an unfortunate coupling.

Could you suggest a graphviz version to move to, based on some testing, and make a separate PR for that?

josephevans
josephevans approved these changes Mar 9, 2022
View reviewed changes
Copy link
Contributor

@josephevans josephevans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@DickJC123 DickJC123 requested a review from szha March 9, 2022 20:22
samskalicky
samskalicky approved these changes Mar 10, 2022
View reviewed changes
Copy link
Contributor

@samskalicky samskalicky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks @DickJC123 !

@DickJC123 DickJC123 merged commit 67467f8 into apache:v1.9.x Mar 10, 2022
DickJC123 added a commit to DickJC123/mxnet that referenced this pull request Mar 12, 2022
@DickJC123
[v1.9.x] [BUGFIX] Upgrade numpy to <1.20.0 to avoid security vulnerab…
042a74f 
…ilities affecting numpy<1.19.1 (apache#20940)

* Pin numpy==1.19.1 to demonstrate issues

* Relax min numpy version

* Make test_np_array_function_protocol delete() testing work with numpy>=1.19

* Fix test_np_delete to also work with numpy>=1.19

* Pip install python module 'packaging'

* More pip-install 'packaging'

* Update windows requirements.txt

* Allow numpy as advanced as 1.19.5
@DickJC123 DickJC123 mentioned this pull request Mar 12, 2022
Merged
6 tasks
DickJC123 added a commit to DickJC123/mxnet that referenced this pull request Mar 14, 2022
@DickJC123
[v1.9.x] [BUGFIX] Upgrade numpy to <1.20.0 to avoid security vulnerab…
87cbd00 
…ilities affecting numpy<1.19.1 (apache#20940)

* Pin numpy==1.19.1 to demonstrate issues

* Relax min numpy version

* Make test_np_array_function_protocol delete() testing work with numpy>=1.19

* Fix test_np_delete to also work with numpy>=1.19

* Pip install python module 'packaging'

* More pip-install 'packaging'

* Update windows requirements.txt

* Allow numpy as advanced as 1.19.5
DickJC123 added a commit that referenced this pull request Mar 18, 2022
@DickJC123
[v1.x] Port #20940 (Upgrade numpy to <1.20.0) from v1.9.x (#20955)
269daba 
* [v1.9.x] [BUGFIX] Upgrade numpy to <1.20.0 to avoid security vulnerabilities affecting numpy<1.19.1 (#20940)

* Pin numpy==1.19.1 to demonstrate issues

* Relax min numpy version

* Make test_np_array_function_protocol delete() testing work with numpy>=1.19

* Fix test_np_delete to also work with numpy>=1.19

* Pip install python module 'packaging'

* More pip-install 'packaging'

* Update windows requirements.txt

* Allow numpy as advanced as 1.19.5

* Trigger CI
DickJC123 added a commit that referenced this pull request Mar 18, 2022
@DickJC123
[v2.0] [BUGFIX] Port #20940 (Upgrade numpy to <1.20.0 ...) from v1.9.x (
95a6a54 
#20957)

* Stop skipping tests mentioned in issue 18600

* Reenable test_np_random_chisquare also

* [v1.9.x] [BUGFIX] Upgrade numpy to <1.20.0 to avoid security vulnerabilities affecting numpy<1.19.1 (#20940)

* Pin numpy==1.19.1 to demonstrate issues

* Relax min numpy version

* Make test_np_array_function_protocol delete() testing work with numpy>=1.19

* Fix test_np_delete to also work with numpy>=1.19

* Pip install python module 'packaging'

* More pip-install 'packaging'

* Update windows requirements.txt

* Allow numpy as advanced as 1.19.5

* Fix test_np_random_{beta,f,chisquare}
@waytrue17 waytrue17 mentioned this pull request Apr 12, 2022
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@marcoabreu marcoabreu Awaiting requested review from marcoabreu marcoabreu is a code owner

@aaronmarkham aaronmarkham Awaiting requested review from aaronmarkham aaronmarkham is a code owner

@szha szha Awaiting requested review from szha

2 more reviewers

@samskalicky samskalicky samskalicky approved these changes

@josephevans josephevans josephevans approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

pr-awaiting-review PR is waiting for code review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@DickJC123 @mxnet-bot @samskalicky @josephevans @mseth10