Incorrect documentation of the hard_sigmoid activation function (TensorFlow 2.16.1)

[Dobiasd]

The documentation of the hard_sigmoid activation function (TensorFlow 2.16.1) states:

The hard sigmoid activation is defined as:

• 0 if if x < -2.5
• 1 if x > 2.5
• 0.2 * x + 0.5 if -2.5 <= x <= 2.5

In reality, it's not -2.5 and 2.5, but -3.0 and 3.0 instead.

Proof:

import tensorflow as tf
for x in range(-40, 40):
    print(x/10, tf.keras.activations.hard_sigmoid(x/10).numpy())

-4.0 0.0
-3.9 0.0
-3.8 0.0
-3.7 0.0
-3.6 0.0
-3.5 0.0
-3.4 0.0
-3.3 0.0
-3.2 0.0
-3.1 0.0
-3.0 0.0
-2.9 0.01666665
-2.8 0.033333343
-2.7 0.049999993
-2.6 0.066666685
-2.5 0.083333336
-2.4 0.09999999
-2.3 0.116666675
-2.2 0.13333333
-2.1 0.15000002
-2.0 0.16666667
-1.9 0.18333334
-1.8 0.2
-1.7 0.21666665
-1.6 0.23333333
-1.5 0.25
-1.4 0.26666668
-1.3 0.28333333
-1.2 0.29999998
-1.1 0.31666666
-1.0 0.33333334
-0.9 0.35
-0.8 0.36666667
-0.7 0.38333333
-0.6 0.4
-0.5 0.41666666
-0.4 0.4333333
-0.3 0.45000002
-0.2 0.46666667
-0.1 0.48333335
0.0 0.5
0.1 0.51666665
0.2 0.53333336
0.3 0.55
0.4 0.56666666
0.5 0.5833333
0.6 0.59999996
0.7 0.6166667
0.8 0.6333333
0.9 0.65000004
1.0 0.6666667
1.1 0.68333334
1.2 0.7
1.3 0.7166667
1.4 0.73333335
1.5 0.75
1.6 0.76666665
1.7 0.7833333
1.8 0.8
1.9 0.81666666
2.0 0.8333333
2.1 0.84999996
2.2 0.8666666
2.3 0.8833334
2.4 0.90000004
2.5 0.9166667
2.6 0.93333334
2.7 0.95
2.8 0.9666667
2.9 0.98333335
3.0 1.0
3.1 1.0
3.2 1.0
3.3 1.0
3.4 1.0
3.5 1.0
3.6 1.0
3.7 1.0
3.8 1.0
3.9 1.0

[SuryanarayanaY]

Hi @Dobiasd ,

Thanks for reporting. I have replicated the issue with TF2.16v which uses keras3. With Tf2.15v it works fine which uses Keras2 . Attached gist for same. Will propose a fix for same.

[SuryanarayanaY]

@Dobiasd , Actually the implementation in Keras3 is inline with torch and jax whereas Keras2 it is different. Changing the documentation accordingly.

[SuryanarayanaY]

Hi @Dobiasd ,

The proposed PR got merged and it will reflect in documentation probably in next TF version release.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No