clean up duplicated errors during training

[erohmensing]

There are certain errors while training that get processed per line, and the repeat a ton when the error shows up a lot:

2019-06-28 17:49:21 WARNING  rasa.core.domain  - Failed to use action 'action_thank_feedback' in history. Please make sure all actions are listed in the domains action list. If you recently removed an action, don't worry about this warning. It should stop appearing after a while.
Processed Story Blocks:   0%|                                                                               | 0/17 [00:00<?, ?it/s, # trackers=1]2019-06-28 17:49:21 WARNING  rasa.core.domain  - Failed to use action 'action_thank_feedback' in history. Please make sure all actions are listed in the domains action list. If you recently removed an action, don't worry about this warning. It should stop appearing after a while.
Processed Story Blocks:   0%|                                                                               | 0/17 [00:00<?, ?it/s, # trackers=2]2019-06-28 17:49:21 WARNING  rasa.core.domain  - Failed to use action 'action_thank_feedback' in history. Please make sure all actions are listed in the domains action list. If you recently removed an action, don't worry about this warning. It should stop appearing after a while.
2019-06-28 17:49:21 WARNING  rasa.core.domain  - Failed to use action 'action_thank_feedback' in history. Please make sure all actions are listed in the domains action list. If you recently removed an action, don't worry about this warning. It should stop appearing after a while.
Processed Story Blocks: 100%|████████████████████████████████████████████████████████████████████| 17/17 [00:00<00:00, 1048.34it/s, # trackers=1]
Processed Story Blocks: 100%|████████████████████████████████████████████████████████████████████████████████| 17/17 [00:00<00:00, 115005.11it/s]
Processed Story Blocks:   0%|                                                                               | 0/17 [00:00<?, ?it/s, # trackers=2]2019-06-28 17:49:21 WARNING  rasa.core.domain  - Failed to use action 'action_thank_feedback' in history. Please make sure all actions are listed in the domains action list. If you recently removed an action, don't worry about this warning. It should stop appearing after a while.
2019-06-28 17:49:21 WARNING  rasa.core.domain  - Failed to use action 'action_thank_feedback' in history. Please make sure all actions are listed in the domains action list. If you recently removed an action, don't worry about this warning. It should stop appearing after a while.

It would be good if we, for example in this case, stored a set of actions for which this error occurs, and then after training dumped all of the warnings (but only one for each action). This also applies to other warnings like "tried to set non-existent slot XXX"

[wochinge]

check if this is still the case

[erohmensing]

I havent been able to get the earlier warning because it only happens after you remove an action from everywhere. However for example in 1.7.0, if you remove a categorical slot that is in your stories from your domain and then try to train (scroll to the right):

Processed Story Blocks:  88%|████████████████████████████████████████████████████████████████████████████████████████████████████▎             | 1864/2118 [00:03<00:00, 493.74it/s, # trackers=1]2020-02-11 10:31:45 ERROR    rasa.core.trackers  - Tried to set non existent slot 'feedback_value'. Make sure you added all your slots to your domain file.
Processed Story Blocks:  88%|████████████████████████████████████████████████████████████████████████████████████████████████████▎             | 1864/2118 [00:03<00:00, 493.74it/s, # trackers=1]2020-02-11 10:31:45 ERROR    rasa.core.trackers  - Tried to set non existent slot 'feedback_value'. Make sure you added all your slots to your domain file.
Processed Story Blocks:  88%|████████████████████████████████████████████████████████████████████████████████████████████████████▎             | 1864/2118 [00:03<00:00, 493.74it/s, # trackers=1]2020-02-11 10:31:45 ERROR    rasa.core.trackers  - Tried to set non existent slot 'feedback_value'. Make sure you added all your slots to your domain file.
Processed Story Blocks:  88%|████████████████████████████████████████████████████████████████████████████████████████████████████▎             | 1864/2118 [00:03<00:00, 493.74it/s, # trackers=1]2020-02-11 10:31:45 ERROR    rasa.core.trackers  - Tried to set non existent slot 'feedback_value'. Make sure you added all your slots to your domain file.
Processed Story Blocks:  88%|████████████████████████████████████████████████████████████████████████████████████████████████████▎             | 1864/2118 [00:03<00:00, 493.74it/s, # trackers=1]2020-02-11 10:31:45 ERROR    rasa.core.trackers  - Tried to set non existent slot 'feedback_value'. Make sure you added all your slots to your domain file.
2020-02-11 10:31:45 ERROR    rasa.core.trackers  - Tried to set non existent slot 'feedback_value'. Make sure you added all your slots to your domain file.
Processed Story Blocks:  88%|████████████████████████████████████████████████████████████████████████████████████████████████████▎             | 1864/2118 [00:03<00:00, 493.74it/s, # trackers=1]2020-02-11 10:31:45 ERROR    rasa.core.trackers  - Tried to set non existent slot 'feedback_value'. Make sure you added all your slots to your domain file.
Processed Story Blocks:  88%|████████████████████████████████████████████████████████████████████████████████████████████████████▎             | 1864/2118 [00:03<00:00, 493.74it/s, # trackers=1]2020-02-11 10:31:45 ERROR    rasa.core.trackers  - Tried to set non existent slot 'feedback_value'. Make sure you added all your slots to your domain file.
Processed Story Blocks:  88%|████████████████████████████████████████████████████████████████████████████████████████████████████▎             | 1864/2118 [00:03<00:00, 493.74it/s, # trackers=1]2020-02-11 10:31:45 ERROR    rasa.core.trackers  - Tried to set non existent slot 'feedback_value'. Make sure you added all your slots to your domain file.
Processed Story Blocks:  88%|████████████████████████████████████████████████████████████████████████████████████████████████████▎             | 1864/2118 [00:03<00:00, 493.74it/s, # trackers=1]2020-02-11 10:31:45 ERROR    rasa.core.trackers  - Tried to set non existent slot 'feedback_value'. Make sure you added all your slots to your domain file.
Processed Story Blocks:  88%|████████████████████████████████████████████████████████████████████████████████████████████████████▎             | 1864/2118 [00:03<00:00, 493.74it/s, # trackers=1]2020-02-11 10:31:45 ERROR    rasa.core.trackers  - Tried to set non existent slot 'feedback_value'. Make sure you added all your slots to your domain file.

[erohmensing]

/Users/ella/rasa/rasa/core/training/dsl.py:422: UserWarning: Found unknown intent 'inform' on line 6. Please, make sure that all intents are listed in your domain yaml.
  docs=DOCS_URL_DOMAINS,
/Users/ella/rasa/rasa/core/training/dsl.py:422: UserWarning: Found unknown intent 'inform' on line 8. Please, make sure that all intents are listed in your domain yaml.
  docs=DOCS_URL_DOMAINS,
/Users/ella/rasa/rasa/core/training/dsl.py:422: UserWarning: Found unknown intent 'inform' on line 21. Please, make sure that all intents are listed in your domain yaml.
  docs=DOCS_URL_DOMAINS,
/Users/ella/rasa/rasa/core/training/dsl.py:422: UserWarning: Found unknown intent 'inform' on line 23. Please, make sure that all intents are listed in your domain yaml.
  docs=DOCS_URL_DOMAINS,
/Users/ella/rasa/rasa/core/training/dsl.py:422: UserWarning: Found unknown intent 'inform' on line 36. Please, make sure that all intents are listed in your domain yaml.
  docs=DOCS_URL_DOMAINS,
/Users/ella/rasa/rasa/core/training/dsl.py:422: UserWarning: Found unknown intent 'inform' on line 38. Please, make sure that all intents are listed in your domain yaml.
  docs=DOCS_URL_DOMAINS,
/Users/ella/rasa/rasa/core/training/dsl.py:422: UserWarning: Found unknown intent 'inform' on line 49. Please, make sure that all intents are listed in your domain yaml.
  docs=DOCS_URL_DOMAINS,

[m-vdb]

@erohmensing @wochinge this sounds super low priority to me. How bothering is this? Are the warnings valid?

[erohmensing]

The warnings are valid, it just throws every time it sees the slot/intent which is kind of annoying. I agree that it's very low, definitely a "nice to have". We can put it on the "help wanted" project instead if you like.

[m-vdb]

ok 👍 added the labels. Thanks @erohmensing !

[erohmensing]

I removed it from the project board and the priority label because there's a separate board for contributors, does that make sense?

[m-vdb]

ahhhh, I thought the label help wanted was enough. Yes it does! 🙏

[m-vdb]

taking a look at this now. They seem to be different kinds of warnings:

• some are sent with logging.warning or even logging.error: for those we need a specific kind of filter to remove duplicates but it won't be perfect, it will only filter duplicates that come one after the other
• some are send using warnings.warn: Python has built-in deduplication for those, it should work