Correct transformer_size parameter if needed for ResponseSelector during loading

[jupyterjazz]

Proposed changes:

• Corrects transformer_size parameter in ResponseSelector which is None by default with a constant integer during model loading
• Otherwise it could lead to errors like (3.0) Error initializing ResponseSelector in rasa test nlu with use_text_as_label: true #10519
• Credits to @mleimeister for an informative and clear analysis

Status (please check what you already did):

• added some tests for the functionality
• updated the documentation
• updated the changelog (please check changelog for instructions)
• reformat files using black (please check Readme for instructions)

[mleimeister]

Does the proposed fix in RasaSequenceLayer also take effect in the training phase of ResponseSelector, not just model loading? In that case the original function ResponseSelector._warn_and_correct_transfomer_size would be redundant and could be removed.

However, if this gets problematic with the ResponseSelector tests (afaik they check that this function is run as expected), it could also be dealt with in a follow-up ticket.

[mleimeister]

For transparency, you could add to the docstring of RasaSequenceLayer._get_transformer_dimensions to clarify in which cases DEFAULT_TRANSFORMER_SIZE is used.

[jupyterjazz]

Does the proposed fix in RasaSequenceLayer also take effect in the training phase of ResponseSelector, not just model loading? In that case the original function ResponseSelector._warn_and_correct_transfomer_size would be redundant and could be removed.

However, if this gets problematic with the ResponseSelector tests (afaik they check that this function is run as expected), it could also be dealt with in a follow-up ticket.

Yes, it does and as you predicted removing the existing correction causes some problems in the tests. I'll create a separate ticket to address that.
Just to clarify, should we remove the whole function or should we leave the warning and disable the correction?

[ancalita]

LGTM 👍🏼

[mleimeister]

Just to clarify, should we remove the whole function or should we leave the warning and disable the correction?

Since the default is then set in RasaSequenceLayer, the warning could be moved there as well and the original function be removed completely. That way, the warning would be logged both when training and loading a model?

[mleimeister]

LGTM 🚀