Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Fix backward_clip num inputs and type of clip params #15688

Merged
merged 6 commits into from
Aug 9, 2019

Conversation

ptrendx
Copy link
Member

@ptrendx ptrendx commented Jul 29, 2019

Description

Fix backward of clip to correctly report 2 inputs instead of 1. Changed the type of a_min and a_max used inside the kernel to avoid numerical inconsistencies in fp16 (even though the final comparison was done in fp32 and the limits were originally in fp32, they were casted to and from fp16 entering the function).

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • Changes are complete (i.e. I finished coding on this PR)
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

@piyushghai
Copy link
Contributor

@ptrendx Can you also add a unit test for the fix you made ?

@mxnet-label-bot Add [Operator, Backend, pr-awaiting-review]

@marcoabreu marcoabreu added Backend Issues related to the backend of MXNet Operator pr-awaiting-review PR is waiting for code review labels Jul 31, 2019
@ptrendx
Copy link
Member Author

ptrendx commented Jul 31, 2019

@piyushghai Sure, I should be able to do it tomorrow.

@ptrendx
Copy link
Member Author

ptrendx commented Aug 2, 2019

Last commit adds the test that uncovered the initial problem and also fixes #12901

@ptrendx ptrendx requested a review from DickJC123 August 5, 2019 19:51
Copy link
Contributor

@DickJC123 DickJC123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to recommend some improvements to the mx.sym.clip documentation:
- correct typo 'Clipping x between a_min and a_x would be` -> 'Clipping x between a_min and a_max would be'
- the description 'clip(x, a_min, a_max) = max(min(x, a_max), a_min))' suggests the definition of MXNet's clip in terms of MXNet's min and max, rather than a mathematical definition. Better to switch to italics as is used in the description of hard_sigmoid.
- the treatment of a_min and a_max is a bit loose. Users may be surprised to learn that values can be passed that exceed the range of a_min and a_max specified in the python code, due to rounding effects. Fixing this won't be particularly concise, so I recommend this for a followup PR if it seems critical to anyone. Best would be to document the behavior with wording like:

The values passed can be as as large as cast(cast(a_max, dtype=float64), dtype=x.dtype),
or as small as  cast(cast(a_min, dtype=float64), dtype=x.dtype), where the casts are round-to-nearest.

@ptrendx
Copy link
Member Author

ptrendx commented Aug 6, 2019

I'm not sure how to not overload the user with information with all those casts there. I made the other changes to docs.

@DickJC123
Copy link
Contributor

My thought with the comment about a_min and a_max casting was to describe precisely the behavior of the op. While my first suggestion might be too detailed, I still think it important to alert the user to the rounding of the limits, in case they have some home-grown bucketing of the clipped outputs. How about:

The actual limits used by clip are the provided a_min and a_max values rounded to the nearest x.dtype.

Copy link
Contributor

@DickJC123 DickJC123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The underlying problem exposed here by clip() with monitor use and exected-vs-actual input counts has now spawned follow-up PR #15834. Given that and the documentation improvements, this PR LGTM.

@ptrendx ptrendx merged commit 0eb213d into apache:master Aug 9, 2019
anirudhacharya pushed a commit to anirudhacharya/mxnet that referenced this pull request Aug 20, 2019
* Fix backward_clip num inputs and type of clip params

* Clip test

* Trigger CI

* Changes to clip docs

* Fix docstring

* Trigger CI
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Backend Issues related to the backend of MXNet Operator pr-awaiting-review PR is waiting for code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants