Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add function to add frequently confused intent to intent report #4899

Merged
merged 34 commits into from
Dec 12, 2019

Conversation

amn41
Copy link
Contributor

@amn41 amn41 commented Dec 4, 2019

Proposed changes:

Adds extra info to the intent_report.json created by rasa test. Specifically, a key is added to the dict which contains up to 2 other intents with high numbers in the confusion matrix.

Needs tests and types and general polish

output looks like this:

  "meta_inform_problem_bad-link": {
    "precision": 0.0,
    "recall": 0.0,
    "f1-score": 0.0,
    "support": 3,
    "confused_with": {
      "faq": 1,
      "affirm": 1
    }
  }

Status (please check what you already did):

  • added some tests for the functionality
  • updated the documentation
  • updated the changelog (please check changelog for instructions)
  • reformat files using black (please check Readme for instructions)

@amn41 amn41 added the type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR label Dec 4, 2019
@amn41 amn41 marked this pull request as ready for review December 4, 2019 13:48
@amn41 amn41 requested a review from wochinge December 4, 2019 14:01
Copy link
Contributor

@wochinge wochinge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool feature 👍

One test with a very small confusion matrix (already there) and one with sth like 5x5 would be great

rasa/nlu/test.py Outdated Show resolved Hide resolved
rasa/nlu/test.py Show resolved Hide resolved
rasa/nlu/test.py Outdated Show resolved Hide resolved
rasa/nlu/test.py Outdated Show resolved Hide resolved
rasa/nlu/test.py Outdated Show resolved Hide resolved
rasa/nlu/test.py Outdated Show resolved Hide resolved
rasa/nlu/test.py Show resolved Hide resolved
rasa/nlu/test.py Outdated Show resolved Hide resolved
rasa/nlu/test.py Outdated Show resolved Hide resolved
rasa/nlu/test.py Outdated Show resolved Hide resolved
@amn41
Copy link
Contributor Author

amn41 commented Dec 5, 2019

thanks @wochinge ! can you please have another quick look?

@amn41 amn41 requested a review from wochinge December 6, 2019 10:07
Copy link
Contributor

@wochinge wochinge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Left a couple of stylistic comments, but otherwise good to go 🚀

tests/nlu/base/test_evaluation.py Outdated Show resolved Hide resolved
tests/nlu/base/test_evaluation.py Outdated Show resolved Hide resolved
tests/nlu/base/test_evaluation.py Outdated Show resolved Hide resolved
tests/nlu/base/test_evaluation.py Outdated Show resolved Hide resolved
tests/nlu/base/test_evaluation.py Outdated Show resolved Hide resolved
tests/nlu/base/test_evaluation.py Outdated Show resolved Hide resolved
tests/nlu/base/test_evaluation.py Outdated Show resolved Hide resolved
tests/nlu/base/test_evaluation.py Show resolved Hide resolved
tests/nlu/base/test_evaluation.py Outdated Show resolved Hide resolved
rasa/nlu/test.py Outdated Show resolved Hide resolved
Copy link
Contributor

@wochinge wochinge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one more comment, but this is now ready to go 👍

rasa/nlu/test.py Outdated Show resolved Hide resolved
rasa/nlu/test.py Show resolved Hide resolved
@amn41 amn41 merged commit 9b937d8 into master Dec 12, 2019
@amn41 amn41 deleted the add-most-confused-intents-to-report branch December 12, 2019 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants