Skip to content

gradient registry has no entry for: Gelu #1913

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
njzjz opened this issue Sep 13, 2022 · 1 comment · Fixed by #1918
Closed

gradient registry has no entry for: Gelu #1913

njzjz opened this issue Sep 13, 2022 · 1 comment · Fixed by #1918
Assignees
Labels
bug reproduced This bug has been reproduced by developers

Comments

@njzjz
Copy link
Member

njzjz commented Sep 13, 2022

LookupError: gradient registry has no entry for: Gelu-- when fiting net OR embbeding activation function is set 'gelu' meanwhile it can be trained in GPU when fiting net AND embbeding activation function is set 'tanh'

Originally posted by @WhiteCrosstheRiver in deepmodeling/dpgen#941 (reply in thread)

Related code:

try:
gelu = tensorflow.nn.gelu
except AttributeError:
@ops.RegisterGradient("Gelu")
def _gelu_cc (op, dy) :
return op_module.gelu_grad(dy, op.inputs[0])
@ops.RegisterGradient("GeluGrad")
def _gelu_grad_cc (op, dy) :
return [op_module.gelu_grad(dy, op.inputs[1]), op_module.gelu_grad_grad(dy, op.inputs[0], op.inputs[1])]

@njzjz njzjz added the bug label Sep 13, 2022
@njzjz njzjz added the reproduced This bug has been reproduced by developers label Sep 15, 2022
@denghuilu
Copy link
Member

#1918 should fix this issue.

@njzjz njzjz linked a pull request Sep 16, 2022 that will close this issue
wanghan-iapcm pushed a commit that referenced this issue Sep 19, 2022
This should avoid the conflicts related to the custom `Gelu` and
Tensorflow's `Gelu`. Also Fix #1913.
@njzjz njzjz closed this as completed Sep 20, 2022
mingzhong15 pushed a commit to mingzhong15/deepmd-kit that referenced this issue Jan 15, 2023
This should avoid the conflicts related to the custom `Gelu` and
Tensorflow's `Gelu`. Also Fix deepmodeling#1913.
ZhengdQin pushed a commit to ZhengdQin/deepmd-kit that referenced this issue Mar 8, 2023
This should avoid the conflicts related to the custom `Gelu` and
Tensorflow's `Gelu`. Also Fix deepmodeling#1913.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug reproduced This bug has been reproduced by developers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants