Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UnexpecTEDIntentPolicy doesn't train on e2e user data #9102

Closed
JEM-Mosig opened this issue Jul 12, 2021 · 1 comment
Closed

UnexpecTEDIntentPolicy doesn't train on e2e user data #9102

JEM-Mosig opened this issue Jul 12, 2021 · 1 comment
Labels
area:rasa-oss 🎡 Anything related to the open source Rasa framework feature:ml/intent-ted type:bug 🐛 Inconsistencies or issues which will cause an issue or problem for users or implementors.

Comments

@JEM-Mosig
Copy link
Contributor

JEM-Mosig commented Jul 12, 2021

Rasa version:
intent-ted branch (#8913), b54a4be

Rasa Version      :         2.7.1
Minimum Compatible Version: 2.6.0
Rasa SDK Version  :         2.7.0
Rasa X Version    :         None
Python Version    :         3.8.0
Operating System  :         macOS-10.16-x86_64-i386-64bit
Python Path       :         /Users/johannes/miniconda3/envs/rasa-master/bin/python

Issue:

When trained on

- story: sad path 1
  steps:
  - intent: greet
  - action: utter_greet
  - user: "blah"
  - bot: "xyz"
  - action: utter_did_that_help
  - intent: affirm
  - action: utter_happy

UnexpecTEDIntentPolicy fails to train.

Error (including full traceback):

Traceback (most recent call last):
  File "/Users/johannes/miniconda3/envs/rasa-master/bin/rasa", line 5, in <module>
    main()
  File "/Users/johannes/rasa/rasa/rasa/__main__.py", line 117, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/Users/johannes/rasa/rasa/rasa/cli/train.py", line 59, in <lambda>
    train_parser.set_defaults(func=lambda args: run_training(args, can_exit=True))
  File "/Users/johannes/rasa/rasa/rasa/cli/train.py", line 91, in run_training
    training_result = train_all(
  File "/Users/johannes/rasa/rasa/rasa/api.py", line 109, in train
    return rasa.utils.common.run_in_loop(
  File "/Users/johannes/rasa/rasa/rasa/utils/common.py", line 296, in run_in_loop
    result = loop.run_until_complete(f)
  File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
  File "/Users/johannes/rasa/rasa/rasa/model_training.py", line 108, in train_async
    return await _train_async_internal(
  File "/Users/johannes/rasa/rasa/rasa/model_training.py", line 288, in _train_async_internal
    await _do_training(
  File "/Users/johannes/rasa/rasa/rasa/model_training.py", line 352, in _do_training
    await _train_core_with_validated_data(
  File "/Users/johannes/rasa/rasa/rasa/model_training.py", line 549, in _train_core_with_validated_data
    await rasa.core.train.train(
  File "/Users/johannes/rasa/rasa/rasa/core/train.py", line 70, in train
    agent.train(training_data, **additional_arguments)
  File "/Users/johannes/rasa/rasa/rasa/core/agent.py", line 753, in train
    self.policy_ensemble.train(
  File "/Users/johannes/rasa/rasa/rasa/core/policies/ensemble.py", line 206, in train
    policy.train(
  File "/Users/johannes/rasa/rasa/rasa/core/policies/ted_policy.py", line 687, in train
    model_data, label_ids = self._prepare_for_training(
  File "/Users/johannes/rasa/rasa/rasa/core/policies/ted_policy.py", line 573, in _prepare_for_training
    tracker_state_features, label_ids, entity_tags = self._featurize_for_training(
  File "/Users/johannes/rasa/rasa/rasa/core/policies/policy.py", line 191, in _featurize_for_training
    state_features, label_ids, entity_tags = self.featurizer.featurize_trackers(
  File "/Users/johannes/rasa/rasa/rasa/core/featurizers/tracker_featurizers.py", line 391, in featurize_trackers
    label_ids = self._convert_labels_to_ids(trackers_as_labels, domain)
  File "/Users/johannes/rasa/rasa/rasa/core/featurizers/tracker_featurizers.py", line 959, in _convert_labels_to_ids
    label_ids = [
  File "/Users/johannes/rasa/rasa/rasa/core/featurizers/tracker_featurizers.py", line 960, in <listcomp>
    [domain.intents.index(intent) for intent in tracker_intents]
  File "/Users/johannes/rasa/rasa/rasa/core/featurizers/tracker_featurizers.py", line 960, in <listcomp>
    [domain.intents.index(intent) for intent in tracker_intents]
ValueError: 'blah' is not in list

Command or request that led to error:

rasa train

Content of configuration file (config.yml) (if relevant):

language: en

pipeline:
  - name: "SpacyNLP"
    model: "en_core_web_md"
  - name: "SpacyTokenizer"
  - name: "SpacyFeaturizer"
  - name: "DIETClassifier"
    entity_recognition: False
    epochs: 1

policies:
  - name: TEDPolicy
    max_history: 5
    epochs: 1
  - name: UnexpecTEDIntentPolicy
    max_history: 5
    epochs: 1
  - name: MemoizationPolicy
  - name: RulePolicy

Content of domain file (domain.yml) (if relevant):
Moodbot domain.

@JEM-Mosig JEM-Mosig added type:bug 🐛 Inconsistencies or issues which will cause an issue or problem for users or implementors. area:rasa-oss 🎡 Anything related to the open source Rasa framework feature:ml/intent-ted labels Jul 12, 2021
@dakshvar22
Copy link
Contributor

@JEM-Mosig Moving it to the IntentTED Backlog as it's a known issue before the release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:rasa-oss 🎡 Anything related to the open source Rasa framework feature:ml/intent-ted type:bug 🐛 Inconsistencies or issues which will cause an issue or problem for users or implementors.
Projects
None yet
Development

No branches or pull requests

2 participants