Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Python BucketingModule bind() with grad_req = 'add' #13984

Merged
merged 2 commits into from
Jan 27, 2019

Conversation

slyforce
Copy link
Contributor

Description

This pull request adds the possibility of changing the grad_req of BucketingModule to values other than the default 'write' in the Python interface.

Previously, grad_req would default to 'write' every time the bucket is switched causing gradient accumulation not to work whenever a non-default bucket key was used.

Also a new unit-test case has been added to test 1. the correct binding of sub-modules and 2. gradient accumulation with different grad_req values.

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

  • BucketingModule now remembers grad_req from bind() call. ,
  • Unit-test for different values of grad_req for BucketingModule.

@slyforce slyforce requested a review from szha as a code owner January 25, 2019 00:24
Copy link
Contributor

@sandeep-krishnamurthy sandeep-krishnamurthy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. LGTM.

@szha - Can you please take a review at this PR?

@sandeep-krishnamurthy sandeep-krishnamurthy added Python pr-awaiting-review PR is waiting for code review labels Jan 25, 2019
@szha szha merged commit bc98c0d into apache:master Jan 27, 2019
jessr92 pushed a commit to jessr92/incubator-mxnet that referenced this pull request Jan 27, 2019
* remember grad_req from bind and apply it to sub-modules

* unit-test for gradient accumulation with bucketing modules
jessr92 pushed a commit to jessr92/incubator-mxnet that referenced this pull request Jan 27, 2019
* remember grad_req from bind and apply it to sub-modules

* unit-test for gradient accumulation with bucketing modules
stephenrawls pushed a commit to stephenrawls/incubator-mxnet that referenced this pull request Feb 16, 2019
* remember grad_req from bind and apply it to sub-modules

* unit-test for gradient accumulation with bucketing modules
haohuanw pushed a commit to haohuanw/incubator-mxnet that referenced this pull request Jun 23, 2019
* remember grad_req from bind and apply it to sub-modules

* unit-test for gradient accumulation with bucketing modules
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
pr-awaiting-review PR is waiting for code review Python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants