Add missing callback hook for optimizer dicts

[edenlightning]

Need a callback hook to consolidate the state of optimizer dicts before saving the final state_dict during model checkpointing.

[ananthsub]

this is blocked on #3545
For more context, I want to use https://github.com/facebookresearch/fairscale/blob/master/fairscale/optim/oss.py with Lightning. I have a callback that defines on_save_checkpoint to call the optimizer's consolidate_state_dict

however, this requires the callback to run in all workers to do the comms before we can save the optimizer state

[stale]

This issue has been automatically marked as stale because it hasn't had any recent activity. This issue will be closed in 7 days if no further activity occurs. Thank you for your contributions, Pytorch Lightning Team!