This repository has been archived by the owner on Nov 17, 2023. It is now read-only.
Add large tensor support binary arithmetic #15785
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
apeforest
approved these changes
Aug 7, 2019
apeforest
reviewed
Aug 7, 2019
apeforest
reviewed
Aug 7, 2019
ChaiBapchya
force-pushed
the
lts_binary_arithmetic
branch
from
0d00648
to
a151181
Compare
apeforest
reviewed
Aug 8, 2019
apeforest
reviewed
Aug 8, 2019
access2rohit
approved these changes
Aug 13, 2019
anirudhacharya
pushed a commit
to anirudhacharya/mxnet
that referenced
this pull request
Aug 20, 2019
* test rdiv * floating_point exception handle * add 10 other ops * added rpow and made numpy consistent * attempt to solve memory issue * linting fix * Trigger notification * lint
access2rohit
pushed a commit
to access2rohit/incubator-mxnet
that referenced
this pull request
Sep 25, 2019
* test rdiv * floating_point exception handle * add 10 other ops * added rpow and made numpy consistent * attempt to solve memory issue * linting fix * Trigger notification * lint
access2rohit
pushed a commit
to access2rohit/incubator-mxnet
that referenced
this pull request
Sep 25, 2019
* test rdiv * floating_point exception handle * add 10 other ops * added rpow and made numpy consistent * attempt to solve memory issue * linting fix * Trigger notification * lint
access2rohit
pushed a commit
to access2rohit/incubator-mxnet
that referenced
this pull request
Sep 25, 2019
* test rdiv * floating_point exception handle * add 10 other ops * added rpow and made numpy consistent * attempt to solve memory issue * linting fix * Trigger notification * lint
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Added 11 binary arithmetic operators -
add,sub,rsub,neg,mul,div,rdiv,mod,rmod,imod,powChecklist
Essentials
Please feel free to remove inapplicable items for your PR.
Design choice
Choice of operator to call
I had a choice between
__op__vsop_symbolvsmx.nd.opSince
x.__add__(y) <=> x+y <=> mx.nd.add(x, y)However, due to cases like these
mod :
x.__mod__(y) <=> x%y <=> mx.nd.modulo(x, y)rmod:
x.__rmod__(y) <=> y%x <=> mx.nd.modulo(y, x)I chose to stick with the
__op__so that the function is consistent.Choice 2
I chose to have separate functions because
a. Easier to debug & test separate operators
b. No 100% 1-to-1 correlation
Divide ops are different
__div__in MXNet vs__truediv__create_2d_tensor vs nd.ones
Chose nd.ones due to performance reasons.
After monitoring multiple runs of test_large_array
Upon running the entire file, it would crash due to lack of memory error.
480Gig machine (dedicated for this one task) - p3.16xl
Reworked the code to ensure
a. variables are reused
b. in-house MXNet function (
mx.nd.ones) used over the previous method (create_2d_tensoruses combination of functions from numpy and mxnet)c. arange is not really needed to test if the function works for large tensor