Currently there is no way to pass an object (not int, float or str) when declaring a set of CategoricalHyperparameter.
Would be very nice to be able to do something like: CategoricalHyperparameter('final_activation', [nn.Softmax(dim=1), nn.Sigmoid()])
Or a callable like CategoricalHyperparameter('final_activation', [lambda : nn.Softmax(dim=4)])
Currently I have to pass a string saying 'softmax', check for that value in the model and create corresponding layer - this would allow to be less verbose while gaining in simplicity.