This repository has been archived by the owner on Nov 17, 2023. It is now read-only.
Showing proper error message when an attempt is made to create large tensor but MXNet is not built with it #16570
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
access2rohit
requested review from
anirudh2290,
eric-haibin-lin and
szha
as code owners
|
@mxnet-label-bot add [pr-work-in-progress] |
access2rohit
force-pushed
the
err_msg_int32
branch
from
9593e8b
to
9e2fbea
Compare
marcoabreu
reviewed
Oct 21, 2019
access2rohit
force-pushed
the
err_msg_int32
branch
from
9e2fbea
to
81bef22
Compare
leezu
reviewed
Oct 21, 2019
access2rohit
force-pushed
the
err_msg_int32
branch
5 times, most recently
from
05c8ac1
to
d035559
Compare
marcoabreu
reviewed
Oct 25, 2019
@marcoabreu I definitely will. But I need to make sure the functionality works fine first. |
|
Sure, sorry for asking prematurely |
apeforest
reviewed
Oct 25, 2019
| @@ -272,10 +272,22 @@ inline bool InitShape(const nnvm::NodeAttrs& attrs, | |||
| CHECK_EQ(in_attrs->size(), 0U); | |||
| CHECK_EQ(out_attrs->size(), 1U); | |||
| mxnet::TShape param_shape = param.shape; | |||
| if (shape_is_known(param_shape) && !features::is_enabled(features::INT64_TENSOR_SIZE)) { | |||
There was a problem hiding this comment.
Why not directly check the compiler flag here?
There was a problem hiding this comment.
I personally prefer that way - looks cleaner :)
There was a problem hiding this comment.
in certain cases shape is not know at the beginning, then param_shape.Size() throws a different error, which is not a good user experience. That's why this check is required.
There was a problem hiding this comment.
@apeforest @marcoabreu do you think this makes sense now ?
access2rohit
force-pushed
the
err_msg_int32
branch
2 times, most recently
from
bf57e72
to
c9c8bb4
Compare
|
@mxnet-label-bot add [pr-awaiting-review] |
|
@apeforest @ChaiBapchya @zheng-da @marcoabreu This PR is ready for review |
access2rohit
changed the title
access2rohit
force-pushed
the
err_msg_int32
branch
2 times, most recently
from
bbd0b30
to
cf7dc07
Compare
marcoabreu
approved these changes
Oct 27, 2019
ChaiBapchya
approved these changes
Oct 27, 2019
access2rohit
force-pushed
the
err_msg_int32
branch
from
40cf110
to
c8f28e0
Compare
apeforest
reviewed
Oct 28, 2019
added 7 commits
October 28, 2019 18:39
…ser attempts to pass large ndarrays as inputs to operators or large shape of inputs for memory allocation.
This reverts commit d035559.
access2rohit
force-pushed
the
err_msg_int32
branch
from
c8f28e0
to
2a7e027
Compare
yajiedesign
pushed a commit
to yajiedesign/mxnet
that referenced
this pull request
Nov 6, 2019
…tensor but MXNet is not built with it (apache#16570) * added error message when creating NDarray and syncing copies to and from CPU * bug fix for 1>>31 -> uint32_t{1}<<31 and showing error message when user attempts to pass large ndarrays as inputs to operators or large shape of inputs for memory allocation. * adding error messages to other operators where user can create NDArray indirectly * bug fix * removing additional checks * Revert "removing additional checks" This reverts commit d035559. * adding tests and comments. Removed int64 check from linspace
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Throwing error message to customers that they need to build MXNet with flag USE_INT64_TENSOR_SIZE=1 when user attempts to create large NDarrays for e.g.
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.
Testing