This repository has been archived by the owner on Nov 17, 2023. It is now read-only.
fix potential floating number overflow, enable float16 #12118
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Why don't you use size_t instead of int32_t? |
|
@larroy Negative number required to mark some indices for special purposes. |
|
ssize_t then? It’s not portable otherwise |
|
@zhreshold did you see my previous comment? Thanks. |
|
@larroy Sorry I missed that. But I don't quite get your point. Isn't int32_t portable enough through stdint standard? |
|
http://pubs.opengroup.org/onlinepubs/7908799/xsh/limits.h.html |
|
CI always fails at deploy stage. Do you have a clue? @larroy |
|
minor problem with doc generation. Are you rebased against master? |
|
@larroy Yep, just rebased 2 hours ago |
|
It's broken in master #11990 |
XinYao1994
pushed a commit
to XinYao1994/incubator-mxnet
that referenced
this pull request
Aug 29, 2018
* fix potential floating number overflow, enable float16 * fix cuda impl * fix cuda imple * fix template substitution for windows * half_f substantiate operand + fix * remove ambiguous operand + for mshadow half_T * fix con't * use int32_t as indices * use overload * try remove ambiguous function overloading * thrust version limit * change sizeof cast from floor to ceil when allocating buffers * cleaner * fix alignment of pointers
anirudh2290
pushed a commit
to anirudh2290/mxnet
that referenced
this pull request
Sep 19, 2018
* fix potential floating number overflow, enable float16 * fix cuda impl * fix cuda imple * fix template substitution for windows * half_f substantiate operand + fix * remove ambiguous operand + for mshadow half_T * fix con't * use int32_t as indices * use overload * try remove ambiguous function overloading * thrust version limit * change sizeof cast from floor to ceil when allocating buffers * cleaner * fix alignment of pointers
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
contrib.box_nms and contrib.bipartite_matchin use floating number array to store indices which can be extremely large.
To avoid overflow when float32/float16 is casted to integer, this PR force use int32_t in indices arrays. (the new limit is 2^31 - 1 instead of 2^k + 1 where k is the mantissa bits of float type)
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.