Skip to content

Conversation

@mrry
Copy link
Contributor

@mrry mrry commented Feb 5, 2021

Description: Log warning when GetGradientForOp() silently fails.

Motivation and Context

  • In some cases, GetGradientForOp() can return without creating any nodes, which may lead to an invalid graph being created.

In some cases, `GetGradientForOp()` can return without creating any nodes, which may lead to an invalid graph being created.
@mrry mrry requested review from SherlockNoMad and wschin February 5, 2021 18:25
@mrry mrry requested a review from a team as a code owner February 5, 2021 18:25
@mrry
Copy link
Contributor Author

mrry commented Feb 5, 2021

@wschin This should go together with your #6581 in case there are other situations where a gradient builder exits without returning any nodes.

The one thing I'm not sure of is whether the gradient graph builder should fail altogether in this case.

weixingzhang
weixingzhang previously approved these changes Feb 5, 2021
@mrry mrry added the training issues related to ONNX Runtime training; typically submitted using template label Feb 5, 2021
@wschin
Copy link
Contributor

wschin commented Feb 10, 2021

A warning looks good to me. Not sure if we should just throw an error.

@mrry mrry merged commit 88d4806 into master Feb 10, 2021
@mrry mrry deleted the mrry-patch-2 branch February 10, 2021 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

training issues related to ONNX Runtime training; typically submitted using template

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants