Log training metrics to visualize them via tensorboard

[tabergma]

Proposed changes:
Related to https://github.com/RasaHQ/research/issues/56

Add option tensorboard_log_directory to DIETClasifier, ResponseSelector and
TEDPolicy.

By default tensorboard_log_directory is None. If a valid directory is provided,
metrics are written during training. After the model is trained you can take a look
at the training metrics in tensorboard. Execute tensorboard --logdir <path-to-given-directory>.

We also write down a model summary (layers, input size, type) in the provided directory.

Status (please check what you already did):

• added some tests for the functionality
• updated the documentation
• updated the changelog (please check changelog for instructions)
• reformat files using black (please check Readme for instructions)

[dakshvar22]

Left some comments. One meta comment: Right now we are logging at an epoch level. I think the default behaviour should be that. Can we add an optional parameter which lets you switch the behaviour to logging at a minibatch level. In my experience, logging at minibatch level can help sometimes, so if it is an easy addition then we should implement it.

[tabergma]

Just to be sure: How would you log on minibatch level? Would you have a counter that increases over time so that you see all steps in one plot? For example, if the first minibatch has 64 steps, the second minibatch would start with step 65 in the plot.

[dakshvar22]

@tabergma Just tried on the scaffold project dataset - The steps for test logger do not correspond to the steps for the train logger. For example, if evaluate_after_every_epochs is set to 20, then the steps for test logger should be 20,40,60 rather than 1,2,3. Sharing the step size would directly give an indication of which epoch does the validation loss correspond to. Otherwise the user has to calculate that on their own. Attaching a screenshot -

If this is easy enough to change, I would suggest we change it now.

[tabergma]

This only happens if you set tensorboard_log_level to minibatch correct? So in that case, training metrics should be plotted after every step, but evaluation metrics just after every epoch?

[dakshvar22]

Yes, setting it to epochs works fine. For your question regarding minibatch - IMO they should be plotted after every evaluate_after_every_epochs steps. So if evaluate_after_every_epochs is set to 2, and number of minibatches in training set is 20 and number of minibatches in validation set is 2 then steps for test logger would be 40,41,80,81,120,121

[tabergma]

Update:
Do you mean something like this?

[dakshvar22]

Yes, there should be some inherent corresponding steps.

[dakshvar22]

Looks good, added two minor comments. I think we should add a simple test as well which atleast checks if the folders are created.

[dakshvar22]

Looks great! 💯 Thanks for adding support for this! 🚀