How do i change ocropus tanh activation function to ReLU

[gotwinn]

Currently researching on which activation function affects ocropus models accuracy

[amitdo]

https://github.com/tmbdev/ocropy/blob/4c4c7d729ceef377c798e85996e1b46412965798/ocrolib/lstm.py#L378

[gotwinn]

So am i correct to say that ocropus uses 2 activation functions, sigmoid and tanh ?

The next question is if i want to change tanh to ReLU how should i go about in doing it ?

[mittagessen]

An lstm cell requires the sigmoid (or rather any differentiable function with an image [0, 1]) for gate activation, the other activation is used for input to hidden state and output, i.e. an LSTM cell uses different activation functions for different parts of the cell.

To change the tanh activations you'll have to switch gfunc/gprime and hfunc/hprime in ocrolib/lstm.py to ReLu and its derivative respectively which should be f'(x) = 1: x>0 if I'm not entirely mistaken.