-
Notifications
You must be signed in to change notification settings - Fork 613
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Typing the gelu activation function. #928
Changes from 20 commits
8c5f509
f144705
6a42b45
038c1a5
912de0b
0092ca9
4a45358
0fc8cd6
e6ce2a1
e82a203
406090f
dced016
dfc46bc
8958ebc
aeba883
54dff52
6000d1f
d1009fa
aebc685
0ef5d0a
0520433
9117054
960fc47
ba688ca
71a161b
7a4822f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -14,13 +14,18 @@ | |
# ============================================================================== | ||
|
||
import tensorflow as tf | ||
from typeguard import typechecked | ||
|
||
from tensorflow_addons.utils import types | ||
from tensorflow_addons.utils.resource_loader import LazySO | ||
|
||
_activation_so = LazySO("custom_ops/activations/_activation_ops.so") | ||
|
||
|
||
@tf.keras.utils.register_keras_serializable(package='Addons') | ||
def gelu(x, approximate=True): | ||
@tf.autograph.experimental.do_not_convert | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmmmm, this has the potential to be a larger issue. To start I'm not sure why this function is being converted with autograph in the first place? Simple op call with no Given that it is for some reason converted, do we see typechecked as forever incompatible with autograph? If so I lean toward not type checking functions that are converting as opposed to the experimental There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
No idea, but it can be easily changed. What's a lot of work to do is the typing. Let's drop the check for now on functions creating the graph. It should be fine for class constructors though since they're not used with autograph. |
||
@typechecked | ||
def gelu(x: types.TensorLike, approximate: bool = True) -> tf.Tensor: | ||
"""Gaussian Error Linear Unit. | ||
|
||
Computes gaussian error linear: | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
unused import?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It reminds me that I need to re-enable this check with flake8