NLG responses not found during NLU pipeline comparison

[indam23]

Rasa version: 1.7.0

Python version: 3.7

Operating system (windows, osx, ...): MacOS

Issue: When running NLU pipeline comparison with a pipeline including the ResponseSelector, NLG responses are not found and training fails. Encountered using rasa-demo repo. This happens because NLG responses are not persisted along with NLU training data after data percentage split.

Error (including full traceback):

2020-02-06 20:43:22 INFO     rasa.nlu.test  - Evaluating configuration 'config' with 25%_exclusion training data.
2020-02-07 07:49:06 WARNING  rasa.nlu.test  - Training model 'config' failed. Error: No response phrases found for out_of_scope/other. Check training data files for a possible wrong intent name in NLU/NLG file

Command or request that led to error:

rasa test nlu --config configs # where configs is a directory of configs

Content of configuration file (config.yml) (if relevant):

Content of domain file (domain.yml) (if relevant):

language: en
pipeline:
- name: WhitespaceTokenizer
- name: CRFEntityExtractor
- name: CountVectorsFeaturizer
  OOV_token: oov
  token_pattern: (?u)\b\w+\b
- name: CountVectorsFeaturizer
  analyzer: char_wb
  min_ngram: 1
  max_ngram: 4
- name: EmbeddingIntentClassifier
  epochs: 50
  ranking_length: 5
- name: DucklingHTTPExtractor
  url: http://localhost:8000
  dimensions:
  - email
  - number
  - amount-of-money
- name: EntitySynonymMapper
- name: ResponseSelector
  retrieval_intent: out_of_scope