Batch normalization module

[JingleiSHI]

Hello,
Thank you very much for your work, that really helps me a lot. I have noticed that the batch normalization function didn't work very well in my models: with batch normalization it will be trained slower and converge at the wrong point. But we I substitute the 'gamma' and 'beta's <truncated_normal_initializer> with <zeros/ones_initializer>, your batch normalization module works very well and converges at the right point. As I am a tensorflow beginner, I don't understand well the differences it brings. Could you please explain to me why you choose these initializers and why they bring such a difference? Thank you very much!

Best wishes,
J. SHI

[tadax]

@JingleiSHI Actually, I have no particular reason for using truncated normal distribution. I not sure why your model doesn't work (zeros/ones initializer are used in tf.layers.batch_normalization).