Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[BUGFIX] Fix #20293 #20462

Merged
merged 7 commits into from
Oct 27, 2021
Merged

[BUGFIX] Fix #20293 #20462

merged 7 commits into from
Oct 27, 2021

Conversation

barry-jin
Copy link
Contributor

@barry-jin barry-jin commented Jul 20, 2021 •
edited
Loading

Description

Fix #20293
state.array_req is already set before allocating the memory.
https://github.com/apache/incubator-mxnet/blob/3480ba2c6df02bb907d3a975d354efa8697c4e71/src/imperative/cached_op.cc#L423-L434

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Comments

  • If this change is a backward incompatible change, why must this change be made.
  • Interesting edge cases to note here

@mxnet-bot
Copy link

Hey @barry-jin , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [centos-cpu, unix-cpu, unix-gpu, windows-cpu, edge, windows-gpu, centos-gpu, sanity, miscellaneous, website, clang]

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jul 20, 2021
ptrendx
ptrendx reviewed Jul 20, 2021
View reviewed changes
src/imperative/cached_op.cc Outdated
state.array_reqs[eid] = reqs[iter->second];
// if ref count is not 0, then we should not assign req user provide
if (reqs[iter->second] == kNullOp && !(ref_count[eid] == 0)) {
state.array_reqs[eid] = kWriteTo;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A question, since I do not really know - do we need to put kWriteTo here? I think it should be handled by the memory pass later on, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we need to put kWriteTo here. Because memory plan already happens in SetBackwardGraph before this. Here we get the updated ref_count from the graph and modify the req_types based on the ref_count.

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Jul 22, 2021
@mseth10 mseth10 added pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jul 22, 2021
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jul 27, 2021
@barry-jin
Copy link
Contributor Author

@mxnet-bot run ci [centos-gpu, unix-gpu]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [centos-gpu, unix-gpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jul 28, 2021
@matteosal
Copy link
Contributor

Is this ready to merge?

@szha
Copy link
Member

szha commented Oct 19, 2021

@barry-jin please add a test and resolve conflict so that we can merge the change.

@barry-jin barry-jin requested a review from szha as a code owner October 19, 2021 18:24
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-awaiting-review PR is waiting for code review pr-awaiting-testing PR is reviewed and waiting CI build and test labels Oct 19, 2021
@mseth10 mseth10 added the pr-work-in-progress PR is still work in progress label Oct 19, 2021
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Oct 19, 2021
@barry-jin
Copy link
Contributor Author

@mxnet-bot run ci [all]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [website, edge, miscellaneous, windows-cpu, unix-cpu, sanity, centos-cpu, clang, centos-gpu, unix-gpu, windows-gpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Oct 19, 2021
@barry-jin
Copy link
Contributor Author

@mxnet-bot run ci [centos-cpu, centos-gpu, unix-cpu, unix-gpu]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [unix-cpu, centos-gpu, centos-cpu, unix-gpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Oct 19, 2021
@barry-jin
Copy link
Contributor Author

@mxnet-bot run ci [unix-cpu]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [unix-cpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Oct 20, 2021
@matteosal
Copy link
Contributor

Can this be merged now?

@szha szha merged commit 026dbf8 into apache:master Oct 27, 2021
@szha
Copy link
Member

szha commented Oct 27, 2021

@barry-jin thanks for the fix!

@matteosal matteosal mentioned this pull request Nov 10, 2021
Open
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@ptrendx ptrendx ptrendx left review comments

@szha szha Awaiting requested review from szha szha is a code owner

Assignees
No one assigned
Labels
pr-awaiting-review PR is waiting for code review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Wrong gradients with C API?
6 participants
@barry-jin @mxnet-bot @matteosal @szha @ptrendx @mseth10