Ignore intent/entity prediction if input is empty

[jupyterjazz]

Proposed changes:

• Some components (e.g. MitieFeaturizer) can't handle empty strings which leads to errors during intent prediction.
• Ignore intent/entity prediction in DIETClassifier, CRFEntityExtractor, ResponseSelector and SklearnIntentClassifier if we encounter an empty string.
• Fixes Empty (or some spaces) input text causes Runtime error #10504

Status (please check what you already did):

• added some tests for the functionality
• updated the changelog (please check changelog for instructions)
• reformat files using black (please check Readme for instructions)

[dakshvar22]

What would happen inside core ensemble (for action prediction) if the input message is empty and hence the intent is missing?

[dakshvar22]

Also, if there was an original issue for which you opened the PR, could you please link it in the PR description?

[jupyterjazz]

What would happen inside core ensemble (for action prediction) if the input message is empty and hence the intent is missing?

In that case, it's considered as nlu_fallback.

I also checked this with a default bot and looking at the trackers_as_states

rasa/rasa/core/featurizers/tracker_featurizers.py

Line 425 in e13bc1a

trackers_as_states = self.prediction_states(

which is used by policies for prediction, empty string gets represented the same way as it would be without this change.
{'user': {'intent': 'nlu_fallback'}, 'prev_action': {'action_name': 'action_listen'}}

[dakshvar22]

Okay, that makes sense. Another question: Looking at the original issue, it seems like the bug only comes up when you add MitieFeaturizer to the config. If the only change that was needed to fix the issue was in DIETClassifier, then why does the issue not come up in the config which does not have MitieFeaturizer in the config (for e.g. - rasa init config).

[jupyterjazz]

then why does the issue not come up in the config which does not have MitieFeaturizer in the config

The reason is that for an empty string MitieFeaturizer doesn't create features which are expected during intent/entity prediction.

[dakshvar22]

Can you please also check if the fix that you added to DIET needs to be added to CRFEntityExtractor and ResponseSelector?

[github-actions]

🚀 A preview of the docs have been deployed at the following URL: https://10604--rasahq-docs-rasa-v2.netlify.app/docs/rasa

[dakshvar22]

Suggested change

	If NLU gets an empty string, ignore predicting intents and entities. Some components can't handle empty strings (e.g `MitieFeaturizer`) which causes errors during intent prediction.
	Fixes a bug which caused machine learning components like `DIETClassifier`, `ResponseSelector`, `CRFEntityExtractor` and `SklearnIntentClassifier` to throw an error when a message with an empty text string was passed to them for processing.

[dakshvar22]

@jupyterjazz Almost there ✨ A couple of things:

1. I see that the original issue was tagged as a bug, so this should go out as a bugfix in a patch release. Hence, the PR should target 3.0.x branch and should be forked from that branch. Currently its main.
2. Once you have done the changes, please run model regression tests overnight so that we are certain that inference is not broken.