Add different swish implementations

[qubvel]

Add different swish implementations:

1. Memory efficient swish

• GPU memory friendly
• Less computationally efficient while training
• Does not supported by torch.jit / torch.onnx

2. Original swish (x * torch.sigmoid(x))

• Less memory efficient
• More computationally efficient while training
• Model can be saved with torch.jit / torch.onnx

Default: memory efficient
Model swish implementation can be changed by .set_swish(memory_efficient=False/True) method

[glenn-jocher]

@qubvel thanks for function! I've tried to implement this in our repo: https://github.com/ultralytics/yolov3, but get worse results (lower mAP and higher loss) when compared to a default Swish() class. Do you know why this might be? See ultralytics/yolov3#441 (comment)

[cswwp]

@qubvel If i train with Memory efficient swish, and exporting model.pt with model.set_swish(memory_efficient=False) + torch.jit.trace(model, example), will it hurt the score?